10 ms·

No surprise, LLM companies optimize for waste. More tokens, and more prompts means more revenue. Reminds of Google’s Prabhakar Raghavan story: deliberately maki

by frevib 1mo ago

No surprise, LLM companies optimize for waste. More tokens, and more prompts means more revenue. Reminds of Google’s Prabhakar Raghavan story: deliberately making search worse [1] [1]: https://pluralistic.net/2024/04/24/naming-names/#prabhakar-raghavan https://pluralistic.net/2024/04/24/naming-names/#prabhakar-r...

nomel 1mo ago
Or, more likely, it's that concise code requires a much deeper, wholistic, understanding that these models just are capable of yet. Same with a junior dev. They don't write long form spaghetti because they're trying to write more LOC. They do it because not doing it is hard, literally above their pay grade. I use LLM every day, but they're still completely awful at architecture. I don't think this clear lack of ability is some conspiracy.
- lilbigdoot 1mo ago
  Personal anecdote: I spent a few days hacking on my compiler to remove 1k lines of code (about 15% of total code) while preserving behavior I was only able to do that after I had solved multiple related problems in different places and started introducing subtle bugs by accident / had difficulty detecting all edge cases I've noticed whenever I use LLMs they introduce the same kind of thing but at much smaller scales than I would. They often suggest solving the wrong problem when I prompt them to diagnose specific bugs too. Usually opting for a shortcut that introduces its own issues and ironically calling the proper direction "too complex" when it's really not.
- frevib 1mo ago
  > I don't think this clear lack of ability is some conspiracy. Maybe currently not. But we will never be able to know, as models are undeterministic and benchmarks are kind of scams. When you cannot prove that something gets worse, rest assured companies will to it.
  - nomel 1mo ago
    This requires something other than a free market, and for China to not exist. We're not there yet. There's plenty of competition to prevent this, for now, with AI spending being an incredibly hot issue.
    - frevib 1mo ago
      Currently the bad quality of architecture and holistic view of models is probably explained by the fact that the models just cannot do it. Free markets and China will make sure of that, kind of. In reality Claude and Codex are better and provide much better tooling, so not much competition from free markets and self-hosted models from China, I think. Then when competition really settles to monoploy or duopoly, like it always does in bigtech, it is really difficult to prove Anthropic and OpenAI enshittify their models. Same with Opus 4.6, that suddenly was worse when 4.7 came out.
      - nomel 1mo ago
        > Currently the bad quality of architecture and holistic view of models is probably explained by the fact that the models just cannot do it. Free markets and China will make sure of that, kind of. I'm not following. Free markets and China will make sure that models stay bad!? Why didn't these entities already stop the progression we've seen? What has been motivating them all this time that has preventing them from stopping? Keep in mind this was all science fiction just a few years ago.
bonoboTP 1mo ago
Bullshit. You can be cynical, it's fine but this is just nonsense. People have so much demand for coding that there is no need to make it waste tokens. People are eager to implement more features, do more testing, more platforms, more file formats, bla bla. There is absolutely no incentive to waste tokens. They can barely serve the demanded tokens anyway. There is in fact incentive to save tokens, so subsidized subscriptions don't consume as much.

by frevib 1mo ago

nomel 1mo ago

Or, more likely, it's that concise code requires a much deeper, wholistic, understanding that these models just are capable of yet. Same with a junior dev. They don't write long form spaghetti because they're trying to write more LOC. They do it because not doing it is hard, literally above their pay grade. I use LLM every day, but they're still completely awful at architecture. I don't think this clear lack of ability is some conspiracy.

lilbigdoot 1mo ago
Personal anecdote: I spent a few days hacking on my compiler to remove 1k lines of code (about 15% of total code) while preserving behavior I was only able to do that after I had solved multiple related problems in different places and started introducing subtle bugs by accident / had difficulty detecting all edge cases I've noticed whenever I use LLMs they introduce the same kind of thing but at much smaller scales than I would. They often suggest solving the wrong problem when I prompt them to diagnose specific bugs too. Usually opting for a shortcut that introduces its own issues and ironically calling the proper direction "too complex" when it's really not.
frevib 1mo ago
> I don't think this clear lack of ability is some conspiracy. Maybe currently not. But we will never be able to know, as models are undeterministic and benchmarks are kind of scams. When you cannot prove that something gets worse, rest assured companies will to it.
- nomel 1mo ago
  This requires something other than a free market, and for China to not exist. We're not there yet. There's plenty of competition to prevent this, for now, with AI spending being an incredibly hot issue.
  - frevib 1mo ago
    Currently the bad quality of architecture and holistic view of models is probably explained by the fact that the models just cannot do it. Free markets and China will make sure of that, kind of. In reality Claude and Codex are better and provide much better tooling, so not much competition from free markets and self-hosted models from China, I think. Then when competition really settles to monoploy or duopoly, like it always does in bigtech, it is really difficult to prove Anthropic and OpenAI enshittify their models. Same with Opus 4.6, that suddenly was worse when 4.7 came out.
    - nomel 1mo ago
      > Currently the bad quality of architecture and holistic view of models is probably explained by the fact that the models just cannot do it. Free markets and China will make sure of that, kind of. I'm not following. Free markets and China will make sure that models stay bad!? Why didn't these entities already stop the progression we've seen? What has been motivating them all this time that has preventing them from stopping? Keep in mind this was all science fiction just a few years ago.

bonoboTP 1mo ago

Bullshit. You can be cynical, it's fine but this is just nonsense. People have so much demand for coding that there is no need to make it waste tokens. People are eager to implement more features, do more testing, more platforms, more file formats, bla bla. There is absolutely no incentive to waste tokens. They can barely serve the demanded tokens anyway. There is in fact incentive to save tokens, so subsidized subscriptions don't consume as much.