8 ms·

Author here. The argument is not that it will keep growing exponentially forever (obviously that is physically impossible), rather that: - given a sustained h

by Inufu 1y ago

Author here. The argument is not that it will keep growing exponentially forever (obviously that is physically impossible), rather that: - given a sustained history of growth along a very predictable trajectory, the highest likelihood short term scenario is continued growth along the same trajectory. Sample a random point on an s-curve and look slightly to the right, what’s the most common direction the curve continues? - exponential progress is very hard to visualize and see, it may appear to hardly make any progress while far away from human capabilities, then move from just below to far above human very quickly

bawolff 1y ago
That feels like you're moving the goal posts a bit. Exponential growth over the short term is very uninteresting. Exponential growth is exciting when it can compound. E.g. if i offered you an investing opportunity 500% / per year compounded daily - that's amazing. If the fine print is that that rate will only last for the very near term (say a week), then it would be worse than a savings account.
- Inufu 1y ago
  Well, growth has been on this exponential already for 5+ years (for the METR eval), and we are at the point where models are very close to matching human expert capabilities in many domains - only one or two more years of growth would put us well beyond that point. Personally I think we'll see way more growth than that, but to see profound impacts on our economy you only need to believe the much more conservative assumption of a little extra growth along the same trend.
  - hnlmorg 1y ago
    Which domains? What I’ve seen is that LLMs are very good at simulating an extremely well read junior. Models know all the tricks but not when to use them. And because of that, you’re continually have to hand hold them. Working with an LLM is really closer to pair programming than it is handing a piece of work to an expert. The stuff I’ve seen in computer vision is far more impressive in terms of putting people out of a job. But even there, it’s still highly specific models left to churn away at tasks that are ostensibly just long and laborious tasks. Which so much of VFX is.
  - ModernMech 1y ago
    > we are at the point where models are very close to matching human expert capabilities in many domains This is not true because experts in these domains don't make the same routine errors LLMs do. You may point to broad benchmarks to prove your point, but actual experts in the benchmarked fields can point to numerous examples of purportedly "expert" LLMs making things up in a way no expert would ever. Expertise is supposed to mean something -- it's supposed to describe both a level of competency and trustworthiness. Until they can be trusted, calling LLMs experts in anything degrades the meaning of expertise.
  - bawolff 1y ago
    > we are at the point where models are very close to matching human expert capabilities in many domains That's a bold claim. I don't think it matches most people's experiences. If that was really true people wouldn't be talking about exponential growth. You don't need exponential growth if you are already almost at your destination.
hnlmorg 1y ago
My point is that the limits of LLMs will be hit long before we they start to take on human capabilities. The problem isn’t that exponential growth is hard to visualise. The problem is that LLMs, as advanced and useful a technique as it is, isn’t suited for AGI and thus will never get us even remotely to the stage of AGI. The human like capabilities are really just smoke and mirrors. It’s like when people anthropomorphisise their car; “she’s being temperamental today”. Except we know the car is not intelligence and it’s just a mechanical problem. Whereas it’s in the AI tech firms best interest to upsell the human-like characteristics of LLMs because that’s how they get VC money. And as we know, building and running models isn’t cheap.
- tim333 1y ago
  There is no particular reason why AI has to stick to language models though. Indeed if you want human like thinking you pretty much have to go beyond language as we do other stuff too if you see what I mean. A recent example: "Google DeepMind unveils its first “thinking” robotics AI" https://arstechnica.com/google/2025/09/google-deepmind-unveils-its-first-thinking-robotics-ai/ https://arstechnica.com/google/2025/09/google-deepmind-unvei...
  - hnlmorg 1y ago
    > There is no particular reason why AI has to stick to language models though. There’s no reason at all. But that’s not the technology that’s in the consumer space, growing exponentially, gaining all the current hype. So at this point in time, it’s just a theoretical future that will happen inevitably but we don’t know when. It could be next year. It could be 10 years. It could be 100 years or more. My prediction is that current AI tech plateaus long before any AGI-capable technology emerges.
    - tim333 1y ago
      Yeah, quite possible.
  - solid_fuel 1y ago
    That's a rather poor choice for an example considering Gemini Robotics-ER is built on a tuned version of Gemini, which is itself an LLM. And while the action model is impressive, the actual "reasoning" here is still being handled by an LLM. From the paper [0]: > Gemini Robotics 1.5 model family. Both Gemini Robotics 1.5 and Gemini Robotics-ER 1.5 inherit Gemini’s multimodal world knowledge. > Agentic System Architecture. The full agentic system consists of an orchestrator and an action model that are implemented by the VLM and the VLA, respectively: > • Orchestrator: The orchestrator processes user input and environmental feedback and controls the overall task flow. It breaks complex tasks into simpler steps that can be executed by the VLA, and it performs success detection to decide when to switch to the next step. To accomplish a user-specified task, it can leverage digital tools to access external information or perform additional reasoning steps. We use GR-ER 1.5 as the orchestrator. > • Action model: The action model translates instructions issued by the orchestrator into low-level robot actions. It is made available to the orchestrator as a specialized tool and receives instructions via open-vocabulary natural language. The action model is implemented by the GR 1.5 model. AI researchers have been trying to discover workable architectures for decades, and LLMs are the best we've got so far. There is no reason to believe that this exponential growth on test scores would or even could transfer to other architectures. In fact, the core advantage that LLMs have here is that they can be trained on vast, vast amounts of text scraped from the internet and taken from pirated books. Other model architectures that don't involve next-token-prediction cannot be trained using that same bottomless data source, and trying to learn quickly from real-world experiences is still a problem we haven't solved. [0] https://storage.googleapis.com/deepmind-media/gemini-robotics/Gemini-Robotics-1-5-Tech-Report.pdf#page=30 https://storage.googleapis.com/deepmind-media/gemini-robotic...
adammarples 1y ago
The most common part of the S-curve by far is the flat bit before and the flat bit after. We just don't graph it because it's boring. Besides which there is no reason at all to assume that this process will follow that shape. Seems like guesswork backed up by hand waving.
- tempfile 1y ago
  Very much handwaving. The question is not meaningful at all without knowing the parameters of the S-curve. It's like saying "I flipped a coin and saw heads. What's the most likely next flip?"
YeGoblynQueenne 1y ago
So it's an argument impossible to counter because it's based on a hypothesis that is impossible to falsify: it predicts that there will either be a bit of progress, or a lot of progress, soon. Well, duh.

by Inufu 1y ago

bawolff 1y ago

That feels like you're moving the goal posts a bit. Exponential growth over the short term is very uninteresting. Exponential growth is exciting when it can compound. E.g. if i offered you an investing opportunity 500% / per year compounded daily - that's amazing. If the fine print is that that rate will only last for the very near term (say a week), then it would be worse than a savings account.

Inufu 1y ago
Well, growth has been on this exponential already for 5+ years (for the METR eval), and we are at the point where models are very close to matching human expert capabilities in many domains - only one or two more years of growth would put us well beyond that point. Personally I think we'll see way more growth than that, but to see profound impacts on our economy you only need to believe the much more conservative assumption of a little extra growth along the same trend.
- hnlmorg 1y ago
  Which domains? What I’ve seen is that LLMs are very good at simulating an extremely well read junior. Models know all the tricks but not when to use them. And because of that, you’re continually have to hand hold them. Working with an LLM is really closer to pair programming than it is handing a piece of work to an expert. The stuff I’ve seen in computer vision is far more impressive in terms of putting people out of a job. But even there, it’s still highly specific models left to churn away at tasks that are ostensibly just long and laborious tasks. Which so much of VFX is.
- ModernMech 1y ago
  > we are at the point where models are very close to matching human expert capabilities in many domains This is not true because experts in these domains don't make the same routine errors LLMs do. You may point to broad benchmarks to prove your point, but actual experts in the benchmarked fields can point to numerous examples of purportedly "expert" LLMs making things up in a way no expert would ever. Expertise is supposed to mean something -- it's supposed to describe both a level of competency and trustworthiness. Until they can be trusted, calling LLMs experts in anything degrades the meaning of expertise.
- bawolff 1y ago
  > we are at the point where models are very close to matching human expert capabilities in many domains That's a bold claim. I don't think it matches most people's experiences. If that was really true people wouldn't be talking about exponential growth. You don't need exponential growth if you are already almost at your destination.

hnlmorg 1y ago

My point is that the limits of LLMs will be hit long before we they start to take on human capabilities. The problem isn’t that exponential growth is hard to visualise. The problem is that LLMs, as advanced and useful a technique as it is, isn’t suited for AGI and thus will never get us even remotely to the stage of AGI. The human like capabilities are really just smoke and mirrors. It’s like when people anthropomorphisise their car; “she’s being temperamental today”. Except we know the car is not intelligence and it’s just a mechanical problem. Whereas it’s in the AI tech firms best interest to upsell the human-like characteristics of LLMs because that’s how they get VC money. And as we know, building and running models isn’t cheap.

tim333 1y ago
There is no particular reason why AI has to stick to language models though. Indeed if you want human like thinking you pretty much have to go beyond language as we do other stuff too if you see what I mean. A recent example: "Google DeepMind unveils its first “thinking” robotics AI" https://arstechnica.com/google/2025/09/google-deepmind-unveils-its-first-thinking-robotics-ai/ https://arstechnica.com/google/2025/09/google-deepmind-unvei...
- hnlmorg 1y ago
  > There is no particular reason why AI has to stick to language models though. There’s no reason at all. But that’s not the technology that’s in the consumer space, growing exponentially, gaining all the current hype. So at this point in time, it’s just a theoretical future that will happen inevitably but we don’t know when. It could be next year. It could be 10 years. It could be 100 years or more. My prediction is that current AI tech plateaus long before any AGI-capable technology emerges.
  - tim333 1y ago
    Yeah, quite possible.
- solid_fuel 1y ago
  That's a rather poor choice for an example considering Gemini Robotics-ER is built on a tuned version of Gemini, which is itself an LLM. And while the action model is impressive, the actual "reasoning" here is still being handled by an LLM. From the paper [0]: > Gemini Robotics 1.5 model family. Both Gemini Robotics 1.5 and Gemini Robotics-ER 1.5 inherit Gemini’s multimodal world knowledge. > Agentic System Architecture. The full agentic system consists of an orchestrator and an action model that are implemented by the VLM and the VLA, respectively: > • Orchestrator: The orchestrator processes user input and environmental feedback and controls the overall task flow. It breaks complex tasks into simpler steps that can be executed by the VLA, and it performs success detection to decide when to switch to the next step. To accomplish a user-specified task, it can leverage digital tools to access external information or perform additional reasoning steps. We use GR-ER 1.5 as the orchestrator. > • Action model: The action model translates instructions issued by the orchestrator into low-level robot actions. It is made available to the orchestrator as a specialized tool and receives instructions via open-vocabulary natural language. The action model is implemented by the GR 1.5 model. AI researchers have been trying to discover workable architectures for decades, and LLMs are the best we've got so far. There is no reason to believe that this exponential growth on test scores would or even could transfer to other architectures. In fact, the core advantage that LLMs have here is that they can be trained on vast, vast amounts of text scraped from the internet and taken from pirated books. Other model architectures that don't involve next-token-prediction cannot be trained using that same bottomless data source, and trying to learn quickly from real-world experiences is still a problem we haven't solved. [0] https://storage.googleapis.com/deepmind-media/gemini-robotics/Gemini-Robotics-1-5-Tech-Report.pdf#page=30 https://storage.googleapis.com/deepmind-media/gemini-robotic...

adammarples 1y ago

The most common part of the S-curve by far is the flat bit before and the flat bit after. We just don't graph it because it's boring. Besides which there is no reason at all to assume that this process will follow that shape. Seems like guesswork backed up by hand waving.

tempfile 1y ago
Very much handwaving. The question is not meaningful at all without knowing the parameters of the S-curve. It's like saying "I flipped a coin and saw heads. What's the most likely next flip?"

YeGoblynQueenne 1y ago

So it's an argument impossible to counter because it's based on a hypothesis that is impossible to falsify: it predicts that there will either be a bit of progress, or a lot of progress, soon. Well, duh.