15 ms·

Yes. The infancy phase of this technology is represented by the pursuit of making wildly grand, wildly expensive, all-purpose models that somehow discern a use

Yes. The infancy phase of this technology is represented by the pursuit of making wildly grand, wildly expensive, all-purpose models that somehow discern a user's full accurate intent from a lazy, underdeveloped, vague idea that they ambiguously and poorly express in a couple dozen words. The adolescence will arrive as those outsized and ill-considered ambitions collapse and we instead see a cambrian explosion of restrained but efficient model+harness-tuples that have been distilled, finetuned, and rigged to deliver on narrowly scoped but idiosyncratically-shaped tasks with incredible efficiency and erogonomics.

jimmaswell 20d ago
This idea has failed to pan out time and time again - people have an instinct that hand-crafted finely-tuned specialized AI systems must be optimal, but throwing more scale and compute to something more generally smart always wins out. It's especially palpable just looking at the last few years of LLM's: a frontier model with all the world knowledge you can stuff in it and every tool at its disposal has always performed the best at all tasks. Suggesting otherwise has become an extraordinary claim requiring extraordinary evidence. http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html Recent comment touching on this in relation to LLM's in more depth: https://news.ycombinator.com/item?id=49322695#49323341 https://news.ycombinator.com/item?id=49322695#49323341
- applfanboysbgon 20d ago
  This idea has not failed to pan out at all. I work for a startup that is exactly what GP described, and am set for life because of how wildly successful it is. Notably, we are successful, in a genuine sense of the word: we bootstrapped from running tiny models to larger and larger models on our own slowly improving fleet of GPUs, and now have millions in revenue without a single dime of outside investment. Conversely, you cannot call taking on ~1 trillion in debt and purchase commitments to scale "success". OpenAI and Anthropic are underwater financially. To be precise, they're in the Mariana Trench.
  - wild_egg 20d ago
    Wait, you actually found a viable counter to The Bitter Lesson? Please say more
    - applfanboysbgon 20d ago
      This is a misunderstanding of either the bitter lesson or what was being claimed, on multiple accounts. Firstly, the bitter lesson is merely about human expertise-tuned algorithms vs. throwing raw compute at a domain. But, notably, it is still domain-specific. No matter how much compute you throw at training an LLM, it is never going to beat a Chess engine at Chess. If you give a Chess engine 1,000,000 compute units and a general-purpose LLM 1,000,000 compute units, the Chess engine is obviously superior at Chess; ergo, there is value in throwing compute units into training models for specific tasks. This is true for within several orders of magnitude of compute, in fact. It's also true that if you give the Chess engine 1000 compute units it'll still beat the all-purpose model with 1,000,000 units, so actually there's a lot of value in training for specific tasks. Secondly, the bitter lesson is predicated on compute being cheap. There was a period where a hand-tuned algorithm informed by human expertise would outperform a raw alpha-beta search at Chess. Then compute got cheaper, and DeepBlue ascended to the top. Compute is now expensive again relative to the tasks being performed. We are absolutely still in a period where human expertise in training LLMs will outperform a naive approach with more raw compute.
      - CamperBob2 20d ago
        I don't know much about chess engines; do they still use hand-tuned algorithms, or are they more like AlphaZero, where they learn through self-play to beat any/all possible human contenders? I don't believe DeepBlue was automated to that extent, but it may have been. In the latter case, the chess example would tend to support the Bitter Lesson, rather than refute it. I would also be VERY slow to claim that general-purpose models will never be competitive at chess. It wasn't so long ago that transformers couldn't add two-digit numbers reliably without resorting to tool use. They are now as good at "mental arithmetic" as any human savant. It wouldn't surprise me at all to see someone come up with a model that just happens to be really, really good at leveraging the portions of its general training data having to do with chess. In fact you could argue that AGI demands such a model, if we are to assume that LLMs are a guidepost in that direction.
- CamperBob2 20d ago
  Suggesting otherwise has become an extraordinary claim requiring extraordinary evidence. VibeThinker 3B constitutes extraordinary evidence, IMO. The first such evidence I've seen myself. Very small model, very low literacy, almost no world knowledge, but it is as good at math and logical reasoning as models a hundred times larger. The Bitter Lesson is a valid and trenchant observation about how about we got here, but I think it's a mistake to assume it tells us very much about where we're going. Too much has changed recently and is still doing so.
  - algo_trader 20d ago
    > VibeThinker 3B constitutes extraordinary evidence.. math and logical reasoning Any similar model aimed at coding? A >10B model for mass spawning/swarming and reporting back to a larger model
    - CamperBob2 20d ago
      To some extent VT 3B is intended as a coding model (see https://old.reddit.com/r/LocalLLaMA/comments/1u7dzdr/scaling_former_vibethinker15b_to_3b_now_it/ https://old.reddit.com/r/LocalLLaMA/comments/1u7dzdr/scaling... ). It reportedly does well on leetcode-type problems, although I didn't check that myself. I wouldn't use it for anything important without heavy supervision, as it's very weak outside its specialty. Not ideal for instruction-following tasks.
  - spockz 20d ago
    So theoretically, if you give that model the means to find information, ascertain the quality of said information, it could still reason its way to an proper answer? Is this whole thing than maybe a read vs write optimisation again? Spent more time and effort training more knowledge into the model upfront and get it out in a single question instead of training a small model and needing more steps to answer the same question?
- nickysielicki 20d ago
  The Bitter Lesson is very popular right now. It seems true right now. It’s having its moment right now. That doesn’t actually mean it’s axiomatically true. Commenter below gets it absolutely correct: stockfish, which runs on your 5 year old phone, is dramatically better at chess than Fable. Like, so much better that it’s not even remotely comparable. The theory of the Bitter Lesson, and it’s only a theory, is that LLMs could eventually outperform stockfish. It’s not true today and it remains to be seen whether it will ever be true. For now, specialized models are absolutely better at specialized tasks.
  - Evidlo 20d ago
    This seems really backwards. The Bitter Lesson is all about large data-based approaches vs hand-crafted ones, it doesn't say anything about language models not trained specifically for chess. I can't find the comment you're referring to, but the latest versions of stockfish are based on neural networks trained on millions of games, so if anything the Bitter Lesson turned out true here.
    - nickysielicki 20d ago
      The conclusion of the bitter lesson would be that a large language model trained on chess commentary as well as being trained on millions of chess games would outperform stockfish which is only trained on millions of chess games. There’s no evidence at this point that this is true.
      - iainmerrick 20d ago
        I think you have it backwards. The common mistake is to think “maybe if we use a blend of raw data and hand-crafted heuristics, we’ll get the best of both worlds!” But the bitter lesson says no, beyond a certain point it’s better just to use the data. Thinking that an LLM might be able to improve on purely “big data” machine learning seems to me to be the same incorrect idea. Its “intelligence” is no more useful than human intelligence. The LLM is based on a massive data corpus, sure, but the amount of data specifically about chess in there pales in comparison to just playing billions of games of chess.
- srcreigh 20d ago
  No. The bitter lesson is about capabilities. GP is talking about efficiency. GP isn’t suggesting that focused narrow model(s) will be more capable than large model, but that many small focused models can have sufficient capability while being more optimal. Also, the bitter lesson is just wrong. The bitter lesson is about hand tuned AI vs computational general methods. However in truth today’s AI uses both. We have general compute heavy models which require narrow expert instructions (eg tools internet docs). LLMs would not be as good without expertly written context, and expert context without LLMs aren’t as good either.
  - anon373839 20d ago
    > The bitter lesson is about hand tuned AI vs computational general methods. However in truth today’s AI uses both. We have general compute heavy models which require narrow expert instructions The models are not even really trained bitter lesson-style anymore. That concept peaked during the era of pre-train scaling, back when it was thought that making a bigger and bigger GPT-3 would automatically solve all problems through prompting. In 2026, the most important part of training is post-training, which uses vast quantities of niche, hand-curated data to fit the models for specific tasks in domains like tax law.
- ZainRiz 20d ago
  I'd respectfully push back on the framing here. If you look at value as purely the LLM output, then there's a valid argument that the best frontier models will always be better than fine tuned specialists. (I'm not convinced personally, but it's a defensible claim) But that misses two dimensions: 1. The cost of acquiring that output 2. What is actually "good enough" for that specialist domain Not every output needs to be the best to produce value. And as specialist models increase in cost, their cost/value proposition goes down. At some point, there's a threshold where cheaper, fine tuned models are "good enough" at the task and also substantially cheaper than the expert models. That's where fine tuning helps. Personally, I became a believer in fine tuning after fine tuning a 1B Qwen model as a second pass over my local voice transcription app, achieving excellent accuracy at ~zero token cost and waaaay lower latency than if I'd invoked my Claude subscription under the hood.
  - DrewADesign 20d ago
    At some point, the idea of cost/benefit analysis in the software business turned into the benefit analysis. The amount of money going into the frontier LLM model game is fantastically ridiculous. Being much better than the free resources doesn’t even touch how much better they will have to be to justify the expense of creating them, let alone continually maintaining these services.
- joefourier 20d ago
  > It's especially palpable just looking at the last few years of LLM's: a frontier model with all the world knowledge you can stuff in it and every tool at its disposal has always performed the best at all tasks. Suggesting otherwise has become an extraordinary claim requiring extraordinary evidence. Absolutely false. At least when it comes to multimodal inputs, even a simple classifier will outperform the largest LLMs who still hallucinate details or don’t describe audio and images accurately. And there’s also the issue of cost/inference speed. Running a trillion parameter model for all tasks will be incredibly costly, require a cloud API, while a tiny CNN can be run locally or at a cost multiple orders of magnitude lower.
- lqstuart 20d ago
  It’s not hard to fine tune a 4B parameter model to outperform GPT-5.x or Claude on a domain-specific task. You don’t even need GRPO or whatever the flavor of the month RL technique is either. I suspect what you’re saying is that massive frontier models are the best for doing big slow tasks on human time horizons—and I emphatically agree—but keep in mind LLMs are useful for quite a bit more than just multi-turn chatbots; and in those instances, throwing infinite compute and latency budgets at a problem is generally not a great way to find the “best” solution :)
- miki123211 20d ago
  I think large-scale systems are optimal when the number of tasks is relatively bounded and costs don't matter. When you're doing something "at scale", you want smaller models, weaker heuristics and rules written in code, as they're much cheaper, much faster, much more predictable and much easier to reason about. If your task is "tell me which pages on the internet are most relevant to this search query", running Claude Fable to evaluate every single page would be ridiculous. You may run something like it at the end, to synthesize information from a small result set for example, as that's a "human-sized" task.
- OtherShrezzing 20d ago
  I think if you start taking cost into account for tasks which models have saturated a long time ago, the bitter lesson stops being applicable. If your task is "write a function to sort a list of 1mn entries alphabetically", a small coding model will exactly match the performance of a frontier model (it'll 1-shot it, with 100% accuracy). So the only differentiator is cost, and the frontier model will cost orders of magnitude more than the specialised model. For most intelligence based tasks, you don't (and never have) needed the tool which "performs best at all tasks". You need the cheapest one which performs adequately for your immediate task. This doesn't mean the bitter lesson is incorrect. At the frontier, it's still correct. It means that it's not applicable at all to lots of tasks.
- schrodinger 19d ago
  If anyone else finds that BitterLesson link difficult to read because it wraps full-width, you can type "javascript:" in your URL bar and paste: document.body.style.cssText='margin:auto;width:600px;' (Chrome strips the "javascript:" for security if I were to include it myself.)
  - xnx 19d ago
    Or use reader mode or resize the browser window.
HoldOnAMinute 20d ago
Someone will eventually figure out how to package it all into a single, cheap chip
- bmitc 20d ago
  That you can then write text to program and make applications with.
theendisney 20d ago
At work i only had early copilot which was hysterically bad at everything. As i wanted it to do the same task repeatedly and could spot wrong results instantly i kept evolving a prompt that attempted to correct all ways it found to do it wrong. It kept inventing new ways to get it wrong until it eventually got it right 90% of the time. My theory is that an avanced model that has no issues with a task could do the prompt enginering much better than i ever could. You could for example run x different queries that all do the same thing and compare the results y times. If there are >1 correct results and the wrong versions are all unique you should be able to drill down to a valid result with even a truly shit model running on a potato. Basically what humans do.
- miki123211 20d ago
  Yes! As long as you have some criteria to judge the final answer, you can do a kind of "prompt-side RLVR", where you have the model generate prompt changes, try a bunch of different prompts and see which ones improve the results. You don't necessarily need a bigger model to do this.
- smartbit 20d ago
  Trying until you find something that doesn’t fail is the model used throughout evolution & engineering, think of bridges collapsed, airplanes falling from the sky or exploded steam locomotives. In engineering you learn from these mistakes and try never making them again. Do we want to go through this evolution every time we solve a software issue? Just because we can with an unlimited number of cheap tokens? I think not, I’d rather use the knowledge build up that also knows about the edge cases forgotten to test. Or better, use multiple models that evaluate each other, as Entropic describes it in their recent report https://news.ycombinator.com/item?id=49316271 https://news.ycombinator.com/item?id=49316271 > We expect that agents coordinating in the wild will act in higher variance ways than we see here, because they’ll have different backgrounds and therefore different contexts. They also, presumably, won’t all be Claudes.
- 59nadir 20d ago
  Early Copilot was tab-complete in editors and was honestly the best version of LLM-assisted development I've used, because it was intentionally small in scope, trivial to verify output from at a glance and easy to opt in and opt out of. I have yet to see anything more useful in terms of code generation; very small-scale code generation (think function-level) comes close, but is a lot more tedious. For analysis, bug hunting, overview and some porting work to popular languages and so on I think the current SotA is fantastic, but they're still very disappointing for code generation past function or small module level.
MASNeo 20d ago
Sounds like mainframes. But then, these never quite disappeared to the date.
noduerme 20d ago
Honestly, I've been perfectly fine using OpenClaw and Claude 4.6 for the last six months, and am used to its flaws. In general it doesn't screw up anything major, but I also wouldn't ask it to reason about anything more than a few code files at the same time. Since I'm not interested in it replacing higher level reasoning or architectural design, it's pretty capable of just doing the modifications and features I suggest. One thing notable is that it makes a difference when I specifically refer to line numbers and variable names when I'm asking it to make changes. Meaning, I also read what it writes and understand what it's doing, and have specific queries where I already know what I'd like to change I add.
eduramirezh 20d ago
it feels like if we had invented hammers, and we're still on the "make them bigger, stronger" phase, but we haven't even invented nails yet.
- panagathon 20d ago
  For large language models, isn't everything nails?

jimmaswell 20d ago

This idea has failed to pan out time and time again - people have an instinct that hand-crafted finely-tuned specialized AI systems must be optimal, but throwing more scale and compute to something more generally smart always wins out. It's especially palpable just looking at the last few years of LLM's: a frontier model with all the world knowledge you can stuff in it and every tool at its disposal has always performed the best at all tasks. Suggesting otherwise has become an extraordinary claim requiring extraordinary evidence. http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html Recent comment touching on this in relation to LLM's in more depth: https://news.ycombinator.com/item?id=49322695#49323341 https://news.ycombinator.com/item?id=49322695#49323341

applfanboysbgon 20d ago
This idea has not failed to pan out at all. I work for a startup that is exactly what GP described, and am set for life because of how wildly successful it is. Notably, we are successful, in a genuine sense of the word: we bootstrapped from running tiny models to larger and larger models on our own slowly improving fleet of GPUs, and now have millions in revenue without a single dime of outside investment. Conversely, you cannot call taking on ~1 trillion in debt and purchase commitments to scale "success". OpenAI and Anthropic are underwater financially. To be precise, they're in the Mariana Trench.
- wild_egg 20d ago
  Wait, you actually found a viable counter to The Bitter Lesson? Please say more
  - applfanboysbgon 20d ago
    This is a misunderstanding of either the bitter lesson or what was being claimed, on multiple accounts. Firstly, the bitter lesson is merely about human expertise-tuned algorithms vs. throwing raw compute at a domain. But, notably, it is still domain-specific. No matter how much compute you throw at training an LLM, it is never going to beat a Chess engine at Chess. If you give a Chess engine 1,000,000 compute units and a general-purpose LLM 1,000,000 compute units, the Chess engine is obviously superior at Chess; ergo, there is value in throwing compute units into training models for specific tasks. This is true for within several orders of magnitude of compute, in fact. It's also true that if you give the Chess engine 1000 compute units it'll still beat the all-purpose model with 1,000,000 units, so actually there's a lot of value in training for specific tasks. Secondly, the bitter lesson is predicated on compute being cheap. There was a period where a hand-tuned algorithm informed by human expertise would outperform a raw alpha-beta search at Chess. Then compute got cheaper, and DeepBlue ascended to the top. Compute is now expensive again relative to the tasks being performed. We are absolutely still in a period where human expertise in training LLMs will outperform a naive approach with more raw compute.
    - CamperBob2 20d ago
      I don't know much about chess engines; do they still use hand-tuned algorithms, or are they more like AlphaZero, where they learn through self-play to beat any/all possible human contenders? I don't believe DeepBlue was automated to that extent, but it may have been. In the latter case, the chess example would tend to support the Bitter Lesson, rather than refute it. I would also be VERY slow to claim that general-purpose models will never be competitive at chess. It wasn't so long ago that transformers couldn't add two-digit numbers reliably without resorting to tool use. They are now as good at "mental arithmetic" as any human savant. It wouldn't surprise me at all to see someone come up with a model that just happens to be really, really good at leveraging the portions of its general training data having to do with chess. In fact you could argue that AGI demands such a model, if we are to assume that LLMs are a guidepost in that direction.
CamperBob2 20d ago
Suggesting otherwise has become an extraordinary claim requiring extraordinary evidence. VibeThinker 3B constitutes extraordinary evidence, IMO. The first such evidence I've seen myself. Very small model, very low literacy, almost no world knowledge, but it is as good at math and logical reasoning as models a hundred times larger. The Bitter Lesson is a valid and trenchant observation about how about we got here, but I think it's a mistake to assume it tells us very much about where we're going. Too much has changed recently and is still doing so.
- algo_trader 20d ago
  > VibeThinker 3B constitutes extraordinary evidence.. math and logical reasoning Any similar model aimed at coding? A >10B model for mass spawning/swarming and reporting back to a larger model
  - CamperBob2 20d ago
    To some extent VT 3B is intended as a coding model (see https://old.reddit.com/r/LocalLLaMA/comments/1u7dzdr/scaling_former_vibethinker15b_to_3b_now_it/ https://old.reddit.com/r/LocalLLaMA/comments/1u7dzdr/scaling... ). It reportedly does well on leetcode-type problems, although I didn't check that myself. I wouldn't use it for anything important without heavy supervision, as it's very weak outside its specialty. Not ideal for instruction-following tasks.
- spockz 20d ago
  So theoretically, if you give that model the means to find information, ascertain the quality of said information, it could still reason its way to an proper answer? Is this whole thing than maybe a read vs write optimisation again? Spent more time and effort training more knowledge into the model upfront and get it out in a single question instead of training a small model and needing more steps to answer the same question?
nickysielicki 20d ago
The Bitter Lesson is very popular right now. It seems true right now. It’s having its moment right now. That doesn’t actually mean it’s axiomatically true. Commenter below gets it absolutely correct: stockfish, which runs on your 5 year old phone, is dramatically better at chess than Fable. Like, so much better that it’s not even remotely comparable. The theory of the Bitter Lesson, and it’s only a theory, is that LLMs could eventually outperform stockfish. It’s not true today and it remains to be seen whether it will ever be true. For now, specialized models are absolutely better at specialized tasks.
- Evidlo 20d ago
  This seems really backwards. The Bitter Lesson is all about large data-based approaches vs hand-crafted ones, it doesn't say anything about language models not trained specifically for chess. I can't find the comment you're referring to, but the latest versions of stockfish are based on neural networks trained on millions of games, so if anything the Bitter Lesson turned out true here.
  - nickysielicki 20d ago
    The conclusion of the bitter lesson would be that a large language model trained on chess commentary as well as being trained on millions of chess games would outperform stockfish which is only trained on millions of chess games. There’s no evidence at this point that this is true.
    - iainmerrick 20d ago
      I think you have it backwards. The common mistake is to think “maybe if we use a blend of raw data and hand-crafted heuristics, we’ll get the best of both worlds!” But the bitter lesson says no, beyond a certain point it’s better just to use the data. Thinking that an LLM might be able to improve on purely “big data” machine learning seems to me to be the same incorrect idea. Its “intelligence” is no more useful than human intelligence. The LLM is based on a massive data corpus, sure, but the amount of data specifically about chess in there pales in comparison to just playing billions of games of chess.
srcreigh 20d ago
No. The bitter lesson is about capabilities. GP is talking about efficiency. GP isn’t suggesting that focused narrow model(s) will be more capable than large model, but that many small focused models can have sufficient capability while being more optimal. Also, the bitter lesson is just wrong. The bitter lesson is about hand tuned AI vs computational general methods. However in truth today’s AI uses both. We have general compute heavy models which require narrow expert instructions (eg tools internet docs). LLMs would not be as good without expertly written context, and expert context without LLMs aren’t as good either.
- anon373839 20d ago
  > The bitter lesson is about hand tuned AI vs computational general methods. However in truth today’s AI uses both. We have general compute heavy models which require narrow expert instructions The models are not even really trained bitter lesson-style anymore. That concept peaked during the era of pre-train scaling, back when it was thought that making a bigger and bigger GPT-3 would automatically solve all problems through prompting. In 2026, the most important part of training is post-training, which uses vast quantities of niche, hand-curated data to fit the models for specific tasks in domains like tax law.
ZainRiz 20d ago
I'd respectfully push back on the framing here. If you look at value as purely the LLM output, then there's a valid argument that the best frontier models will always be better than fine tuned specialists. (I'm not convinced personally, but it's a defensible claim) But that misses two dimensions: 1. The cost of acquiring that output 2. What is actually "good enough" for that specialist domain Not every output needs to be the best to produce value. And as specialist models increase in cost, their cost/value proposition goes down. At some point, there's a threshold where cheaper, fine tuned models are "good enough" at the task and also substantially cheaper than the expert models. That's where fine tuning helps. Personally, I became a believer in fine tuning after fine tuning a 1B Qwen model as a second pass over my local voice transcription app, achieving excellent accuracy at ~zero token cost and waaaay lower latency than if I'd invoked my Claude subscription under the hood.
- DrewADesign 20d ago
  At some point, the idea of cost/benefit analysis in the software business turned into the benefit analysis. The amount of money going into the frontier LLM model game is fantastically ridiculous. Being much better than the free resources doesn’t even touch how much better they will have to be to justify the expense of creating them, let alone continually maintaining these services.
joefourier 20d ago
> It's especially palpable just looking at the last few years of LLM's: a frontier model with all the world knowledge you can stuff in it and every tool at its disposal has always performed the best at all tasks. Suggesting otherwise has become an extraordinary claim requiring extraordinary evidence. Absolutely false. At least when it comes to multimodal inputs, even a simple classifier will outperform the largest LLMs who still hallucinate details or don’t describe audio and images accurately. And there’s also the issue of cost/inference speed. Running a trillion parameter model for all tasks will be incredibly costly, require a cloud API, while a tiny CNN can be run locally or at a cost multiple orders of magnitude lower.
lqstuart 20d ago
It’s not hard to fine tune a 4B parameter model to outperform GPT-5.x or Claude on a domain-specific task. You don’t even need GRPO or whatever the flavor of the month RL technique is either. I suspect what you’re saying is that massive frontier models are the best for doing big slow tasks on human time horizons—and I emphatically agree—but keep in mind LLMs are useful for quite a bit more than just multi-turn chatbots; and in those instances, throwing infinite compute and latency budgets at a problem is generally not a great way to find the “best” solution :)
miki123211 20d ago
I think large-scale systems are optimal when the number of tasks is relatively bounded and costs don't matter. When you're doing something "at scale", you want smaller models, weaker heuristics and rules written in code, as they're much cheaper, much faster, much more predictable and much easier to reason about. If your task is "tell me which pages on the internet are most relevant to this search query", running Claude Fable to evaluate every single page would be ridiculous. You may run something like it at the end, to synthesize information from a small result set for example, as that's a "human-sized" task.
OtherShrezzing 20d ago
I think if you start taking cost into account for tasks which models have saturated a long time ago, the bitter lesson stops being applicable. If your task is "write a function to sort a list of 1mn entries alphabetically", a small coding model will exactly match the performance of a frontier model (it'll 1-shot it, with 100% accuracy). So the only differentiator is cost, and the frontier model will cost orders of magnitude more than the specialised model. For most intelligence based tasks, you don't (and never have) needed the tool which "performs best at all tasks". You need the cheapest one which performs adequately for your immediate task. This doesn't mean the bitter lesson is incorrect. At the frontier, it's still correct. It means that it's not applicable at all to lots of tasks.
schrodinger 19d ago
If anyone else finds that BitterLesson link difficult to read because it wraps full-width, you can type "javascript:" in your URL bar and paste: document.body.style.cssText='margin:auto;width:600px;' (Chrome strips the "javascript:" for security if I were to include it myself.)
- xnx 19d ago
  Or use reader mode or resize the browser window.

HoldOnAMinute 20d ago

Someone will eventually figure out how to package it all into a single, cheap chip

bmitc 20d ago
That you can then write text to program and make applications with.

theendisney 20d ago

At work i only had early copilot which was hysterically bad at everything. As i wanted it to do the same task repeatedly and could spot wrong results instantly i kept evolving a prompt that attempted to correct all ways it found to do it wrong. It kept inventing new ways to get it wrong until it eventually got it right 90% of the time. My theory is that an avanced model that has no issues with a task could do the prompt enginering much better than i ever could. You could for example run x different queries that all do the same thing and compare the results y times. If there are >1 correct results and the wrong versions are all unique you should be able to drill down to a valid result with even a truly shit model running on a potato. Basically what humans do.

miki123211 20d ago
Yes! As long as you have some criteria to judge the final answer, you can do a kind of "prompt-side RLVR", where you have the model generate prompt changes, try a bunch of different prompts and see which ones improve the results. You don't necessarily need a bigger model to do this.
smartbit 20d ago
Trying until you find something that doesn’t fail is the model used throughout evolution & engineering, think of bridges collapsed, airplanes falling from the sky or exploded steam locomotives. In engineering you learn from these mistakes and try never making them again. Do we want to go through this evolution every time we solve a software issue? Just because we can with an unlimited number of cheap tokens? I think not, I’d rather use the knowledge build up that also knows about the edge cases forgotten to test. Or better, use multiple models that evaluate each other, as Entropic describes it in their recent report https://news.ycombinator.com/item?id=49316271 https://news.ycombinator.com/item?id=49316271 > We expect that agents coordinating in the wild will act in higher variance ways than we see here, because they’ll have different backgrounds and therefore different contexts. They also, presumably, won’t all be Claudes.
59nadir 20d ago
Early Copilot was tab-complete in editors and was honestly the best version of LLM-assisted development I've used, because it was intentionally small in scope, trivial to verify output from at a glance and easy to opt in and opt out of. I have yet to see anything more useful in terms of code generation; very small-scale code generation (think function-level) comes close, but is a lot more tedious. For analysis, bug hunting, overview and some porting work to popular languages and so on I think the current SotA is fantastic, but they're still very disappointing for code generation past function or small module level.

MASNeo 20d ago

Sounds like mainframes. But then, these never quite disappeared to the date.

noduerme 20d ago

Honestly, I've been perfectly fine using OpenClaw and Claude 4.6 for the last six months, and am used to its flaws. In general it doesn't screw up anything major, but I also wouldn't ask it to reason about anything more than a few code files at the same time. Since I'm not interested in it replacing higher level reasoning or architectural design, it's pretty capable of just doing the modifications and features I suggest. One thing notable is that it makes a difference when I specifically refer to line numbers and variable names when I'm asking it to make changes. Meaning, I also read what it writes and understand what it's doing, and have specific queries where I already know what I'd like to change I add.

eduramirezh 20d ago

it feels like if we had invented hammers, and we're still on the "make them bigger, stronger" phase, but we haven't even invented nails yet.

panagathon 20d ago
For large language models, isn't everything nails?