9 ms·
Running local LLMs offline on a ten-hour flight
- asimovDev 5mo agoMy 14in M3 Max turns the fans way up when running local agentic coding for me to be comfortable with using it in a public place.
- gck1 5mo agoI don't use my M3 Max in public, but running local models on it makes me uncomfortable because of fans too. I also have a Dell laptop that spin up fans if I open a text editor, and it feels normal, but on Mac, I feel like fans spinning up is me somehow abusing it and shortening its lifespan.
- aitchnyu 5mo agoReminds me of a previous generation (plastic shell) MBP drawing both AC and battery for sustained peak performance, fans spinning.
- calcexp 5mo agoI don't know if this has been asked here, but is it really possible to connect to a free search service like Google? The point is to use information that's relevant to, say, 2026.
- prettyblocks 5mo agoThere are paid solutions for this, I've seen Exa recommended.
- prettyblocks 5mo agoI've been relatively impressed with local llms, but by far the hardest thing for me is how hot they make my laptop run.
- devsecopsify 5mo agoAuthor here Great engagement! Apologies for the sloppinnes in model names. I've been writing it at 1am in Vegas, which was 9am in London (a time my body was operating at) - I was dozing off while finishing it - but excitement about my offline experience kept be _somewhat_ going. I've fixed model naming inconsistencies. I've updated post with some more info and responses to your comments, I hope you find it useful.
- ddarolfi 5mo agoQwen 4.6 36B? Do they mean Qwen3.6-35B-A3B?
- mikeatlas 5mo agoyes
- Johnny_Bonk 5mo agoSo I have a RTX 3080 10GB VRAM which I've been using with Qwen2.5 Coder and Gemma 4 E2B. Im wondering what models you have tried with which quants.
- trvz 5mo agoYes. The author is really sloppy if that wasn’t clear from the article.
- deanc 5mo agoThis has been exactly my experience too. I've tried multiple harnesses (pi, claude code, codex) with multiple variants of qwen3.6 and gemma4 driven by both o mlx and ollama - and every single time I try to do anything meaningful I end up in a loop. On a 64GB Macbook Pro M3 Max. I really don't know what the hell people are doing locally, and suspect a lot of the hype around running these models locally is bullshit. Sure, you can make it do something but certainly nothing useful or substantial.
- mtrifonov 5mo ago[flagged]
- rgovostes 5mo agoAnd was this written by a local model or a frontier cloud one?
- ryandrake 5mo agoSame here. Every time a new local model comes out, I give it a spin with a pretty vanilla coding task ("refactor this method to take two parameters instead of one", or "fix this class of compiler warning across the ~20 file codebase") and more often than not, they get in endless loops, or fail in very unusual ways. They don't yet even approach the usefulness of SOTA models. It's obviously not a fair comparison, though. My 20GB GPU is never going to beat whatever enormous backend Google or Anthropic have.
- 2ndorderthought 5mo agoYou can do this with really small models but you have to do a more legwork. I wouldn't expect most trivially small models to handle anything more than 1 file reliably. The new qwen 3.6 is different though, I have heard cases where it is behaving close to sonnet. That said I don't see why people are so scared to touch code even if it saves them 500 euro a month. Using my IDEs find across my repo and auto replacing 2 patterns is trivial to do and way faster to do by hand. I mostly use small models, it prevents a lot of the issues I've seen with large models and vibe/agentic coding medium to long term. I also write a lot of code.
- vladgur 5mo agoThat window seat with the 14” laptop seems extremely claustrophobic. That’s the real limitation on an economy flight - space rather than power or the internet… at least it would be for me. The only times I was able to get my laptop out and do some productive work was when I either was sitting in premium economy isle seat with room to spare or when there was an empty seat next to me
- rootusrootus 5mo agoI'd probably choose the window seat myself, because while it is cramped, it is predictably so. When I sit in an aisle seat, it's not as cramped but I regularly get shoulder checked by passing people or beverage carts. What really makes me nervous if I'm in an economy seat is the seat in front of me. Depending on how the seat is designed, if the person suddenly reclines (or hell, just flexes the seat a bunch while moving around), it can come pretty close to pinching the laptop screen. That would be bad news.
- ryandrake 5mo agoThat was the first thing I thought of when I saw the image. That's a very expensive computer that you risk destroying when the 300lb guy in front of you decides to lean back. The ergonomics of using a laptop on an economy-class tray table are not worth it. You're sitting there like a T-rex trying to make your arms as small as possible to tap on the keys. And the vertical viewing angle to your screen sometimes prevents you from even seeing anything. I wouldn't even bring my laptop out during a flight.
- democracy 5mo agoYou just learn how to type with your toes. It is an easy skill to master and then comes handy when you drive, or sit at important meetings (you can just keep coding away) and noone's the wiser.
- walthamstow 5mo agoIn the image it's on his lap, not the tray table. I agree, using the tray is not worth it. The ideal is a tray that folds in half so I can use that to hold a drink and keep the machine on my lap. The tradeoff of poor comfort is insane productivity, for me anyway. Being restricted in place, no wifi, inconvenient toilet breaks, not in control of meal times, all means I get a lot of work done
- bobro 5mo agoCan’t you guys just read a book and take a nap?
- 3form 5mo agoI suppose the ones that do, wouldn't consider such a turn of events postworthy.
- cpursley 5mo agoI'm jealous of people who can actually get comfortable enough to sleep on flights.
- stavros 5mo agoWhy would I do that when making things is so much fun?
- deleted 5mo ago[deleted]
- mdni007 5mo agoBut then how can I show random people how productive I am?
- ducttape12 5mo agoYeah, for real. Imagine being so addicted to the AI slot machine that you can't be without it for 10 hours.
- j1000 5mo agoTo be honest, I think possibility to work and travel is con rather than perk of current times.
- ryandrake 5mo agoIt hit different at different points in my life. When I was in my 20s I thought "Wow! I get to go on an international trip to a place I've never been, and work is paying for everything?!? I'll go whenever you need me to go!" Now that I'm almost 50, it's "Fuck. Another 14 hour international flight, to somewhere I'll likely only have time to see the inside of two buildings. What's the local language again? Do I drive on the left or right? Wait, how long do I need to stay? Please no."
- HoldOnAMinute 5mo agoThey keep removing the ability for you to have any downtime.
- jamesu 5mo agoI'm not a huge fan of working while travelling, it's way too distracting while trying to do intricate development work.
- bilekas 5mo agoTrying LLM in the air with a 6.200 EUR laptop... Sorry if it's not exactly relatable..
- zahlman 5mo agoYou got downvoted, but my reaction was much the same honestly.
- bilekas 5mo agoI don't mind being downvoted, if we all had the same opinion it would be quite boring around here. I guess I'm just jaded from AI news.
- dgacmu 5mo agoone one hand, I'm typing this on a 5 year old M1 MBP that I still can't bring myself to replace because it just continues to do the things I need; on the other hand, $6200 every few years is pretty tiny compared to a typical US developer salary, so is this really that crazy if it's your primary work machine?
- builderminkyu 5mo agotried doing exactly this with ollama on a cross-country flight last month. my macbook basically turned into a jet engine and the battery died in under an hour. curious if you had to heavily throttle the cpu or stick to super small quants (like 4 bit phi3) to actually make it through 10 hours without a power outlet?
- julianlam 5mo agoUsing an LLM to code without AC power is ... certainly a choice. On my framework 13 a couple exploratory prompts and a code writing step drained my battery to 85% in about 5 minutes.
- tamimio 5mo agoCan’t wait for more people to do the same and eventually getting laptops banned on board due fear of catching fire..
- scastiel 5mo agoInteresting, I did and document the same kind of experiment a few months ago [1], it looks like so much changed since then! [1] https://betweentheprompts.com/40000-feet/ https://betweentheprompts.com/40000-feet/
- mumbisChungo 5mo ago>Qwen 4.6 36B Did the author mean Qwen3.6-27B? Qwen3.6-35B-A3B?
- walrus01 5mo agoAs much as it's a fun gimmick to run a relatively good sized LLM like qwen 3.6 35B locally, I would much rather have the ability to run it remotely on a piece of hardware I control via VPN session. Much better on battery life and heat. If I'm on an airplane I care about having as much battery life as possible. Let's say you have a basic setup like llama.cpp and llama-server on a remote server (even if it's just sitting under your home office desk) running a 35GB Q8 quantized model of qwen 3.6 35B, it's not difficult to make llama-server available to your laptop over just about any form of internet connection and VPN. Having the ability to run that same model locally if you really need to because no internet connection whatsoever is available, but the times that you simultaneously have no internet and a serious need for something the model can output are fairly rare these days.
- datadrivenangel 5mo agoAgreed. I got a beefy M5 MBP for local llms and for sustained inference it gets hot enough that I worry it may end up shortening its life.
- garethsprice 5mo agoThis is what I am doing - it is rare that I'm in a situation with no Internet while traveling, but very often there is an intermittent connection. Using local models or even hosted foundation models is frustrating exercise in cancelled jobs and timeouts, but Tailscale + mosh + tmux is a really nice way to connect to a workstation and resume from where the session left off - or leave it running doing its thing and come back to it later. Same with running my local dev environment's docker containers, now they run on that workstation and my battery life is far higher, treating my portable device as a dumb terminal.
- seattle_spring 5mo agoWith more and more flights offering Starlink, I don't see why this would really ever be necessary. Also, agreed with the other commenters: just read a damn book and take a nap.
- AntiUSAbah 5mo agoI hope starlink will not be the default. Not only shouldn't we support someone like Elon Musk but also don't you find it hypcritic to respond with 'just read a damn book' and suggesting starlink?
- JDPy 5mo ago[dead]
- cube00 5mo agoMacBook cable: 94W delivered Return flight will test this with the correct cable. I expect at least 16% improvement against the 70W cap Some plane sockets cut out completely if you attempt to draw more then the limit rather then continuing to provide power at the limit.
- djsavvy 5mo agoI’ve long suspected as such. Is there a way to power limit what my macbook pulls from the outlet?
- davidcox143 5mo agoThe MBP also throttles surprisingly easily. I have a 16 MBP and ended up buying a cooling stand that uses a 20w peltier cooler (solid state heat pump). It’s fixed the throttling completely, although I’m somewhat nervous about forgetting to turn the cooler off and creating condensation inside the case…
- ntcho 5mo agoHave you tried MacsFanControl? MBP fans rarely turn on by themselves, but controlling yourself makes throttling not a problem
- itsuckslol 5mo ago[dead]
- yjadsfgasdf 5mo ago[dead]