7 ms·

"There is no evidence that LLMs are the roadmap to AGI." - There's plenty of evidence. What do you think the last few years have been all about? Hell, GPT-4 wou

"There is no evidence that LLMs are the roadmap to AGI." - There's plenty of evidence. What do you think the last few years have been all about? Hell, GPT-4 would already have qualified as AGI about a decade ago.

idiotsecant 2y ago
Have you ever heard of a local maxima? You don't get an attack helicopter by breeding stronger and stronger falcons.
- lolinder 2y ago
  For an industry that spun off of a research field that basically revolves around recursive descent in one form or another, there's a pretty silly amount of willful ignorance about the basic principles of how learning and progress happens. The default assumption should be that this is a local maximum, with evidence required to demonstrate that it's not. But the hype artists want us all to take the inevitability of LLMs for granted—"See the slope? Slopes lead up! All we have to do is climb the slope and we'll get to the moon! If you can't see that you're obviously stupid or have your head in the sand!"
  - zmgsabst 2y ago
    You’re implicitly assuming only a global maximum will lead to useful AI. There might be many local maxima that cross the useful AI or even AGI threshold.
    - eru 2y ago
      And we aren't even at a local maximum. There's still plenty of incremental upwards progress to be made.
    - lolinder 2y ago
      I never said anything about usefulness, and it's frustrating that every time I criticize AGI hype people move the goalposts and say "but it'll still be useful!" I use GitHub Copilot every day. We already have useful "AI". That doesn't mean that the whole thing isn't super overhyped.
  - int_19h 2y ago
    So far we haven't even climbed this slope to the top yet. Why don't we start there and see if it's high enough or not first? If it's not, at the very least we can see what's on the other side, and pick the next slope to climb. Or we can just stay here and do nothing.
gwervc 2y ago
No, GPT-4 would have been classified as it is today: a (good) generator of natural language. While this is a hard classical NLP task, it's a far cry from intelligence.
- falcor84 2y ago
  GPT-4 is a good generator of natural language in the same sense that Google is a good generator of ip packets.
coldtea 2y ago
>What do you think the last few years have been all about? Next token language-based predictors with no more intelligence than brute force GIGO which parrot existing human intelligence captured as text/audio and fed in the form of input data. 4o agrees: "What you are describing is a language model or next-token predictor that operates solely as a computational system without inherent intelligence or understanding. The phrase captures the essence of generative AI models, like GPT, which rely on statistical and probabilistic methods to predict the next piece of text based on patterns in the data they’ve been trained on"
- thrwthsnw 2y ago
  Everything you said is parroting data you’ve trained on, two thirds of it is actual copy paste
  - coldtea 2y ago
    >Everything you said is parroting data you’ve trained on "Just like" an LLM, yeah sure... Like how the brain was "just like" a hydraulic system (early industrial era), like a clockwork with gears and differentiation (mechanical engineering), "just like" an electric circuit (Edison's time), "just like" a computer CPU (21st century), and so on... You're just assuming what you should prove
  - mrbungie 2y ago
    He probably didn't need petabytes of reddit posts and millions of gpu-hours to parrot that though. I still don't buy the "we do the same as LLMs" discourse. Of course one could hypothesize the human brain language center may have some similarities to LLMs, but the differences in resource usage and how those resources are used to train humans and LLMs are remarkable and may indicate otherwise.
    - shwouchk 2y ago
      Not text, he had petabytes of video, audio, and other sensory inputs. Heck, a baby sees petabytes of video before first word is spoken And he probably cant quote Shakespeare as well ;)
n144q 2y ago
> GPT-4 would already have qualified as AGI about a decade ago. Did you just make that up?
- OtomotO 2y ago
  Probably asked an "AI"
- wat10000 2y ago
  A lot of people held that passing the Turing Test would indicate human-level intelligence. GPT-4 passes.
  - bigpingo 2y ago
    Link to GPT-4 passing the turing test? Tried googling, could not find anything.
    - wat10000 2y ago
      Google must be really going downhill. DDG “gpt turing test” provides nothing but relevant links. Here’s a paper: https://arxiv.org/pdf/2405.08007 https://arxiv.org/pdf/2405.08007
OtomotO 2y ago
The last four years? ELIZA 2.0

idiotsecant 2y ago

Have you ever heard of a local maxima? You don't get an attack helicopter by breeding stronger and stronger falcons.

lolinder 2y ago
For an industry that spun off of a research field that basically revolves around recursive descent in one form or another, there's a pretty silly amount of willful ignorance about the basic principles of how learning and progress happens. The default assumption should be that this is a local maximum, with evidence required to demonstrate that it's not. But the hype artists want us all to take the inevitability of LLMs for granted—"See the slope? Slopes lead up! All we have to do is climb the slope and we'll get to the moon! If you can't see that you're obviously stupid or have your head in the sand!"
- zmgsabst 2y ago
  You’re implicitly assuming only a global maximum will lead to useful AI. There might be many local maxima that cross the useful AI or even AGI threshold.
  - eru 2y ago
    And we aren't even at a local maximum. There's still plenty of incremental upwards progress to be made.
  - lolinder 2y ago
    I never said anything about usefulness, and it's frustrating that every time I criticize AGI hype people move the goalposts and say "but it'll still be useful!" I use GitHub Copilot every day. We already have useful "AI". That doesn't mean that the whole thing isn't super overhyped.
- int_19h 2y ago
  So far we haven't even climbed this slope to the top yet. Why don't we start there and see if it's high enough or not first? If it's not, at the very least we can see what's on the other side, and pick the next slope to climb. Or we can just stay here and do nothing.

gwervc 2y ago

No, GPT-4 would have been classified as it is today: a (good) generator of natural language. While this is a hard classical NLP task, it's a far cry from intelligence.

falcor84 2y ago
GPT-4 is a good generator of natural language in the same sense that Google is a good generator of ip packets.

coldtea 2y ago

>What do you think the last few years have been all about? Next token language-based predictors with no more intelligence than brute force GIGO which parrot existing human intelligence captured as text/audio and fed in the form of input data. 4o agrees: "What you are describing is a language model or next-token predictor that operates solely as a computational system without inherent intelligence or understanding. The phrase captures the essence of generative AI models, like GPT, which rely on statistical and probabilistic methods to predict the next piece of text based on patterns in the data they’ve been trained on"

thrwthsnw 2y ago
Everything you said is parroting data you’ve trained on, two thirds of it is actual copy paste
- coldtea 2y ago
  >Everything you said is parroting data you’ve trained on "Just like" an LLM, yeah sure... Like how the brain was "just like" a hydraulic system (early industrial era), like a clockwork with gears and differentiation (mechanical engineering), "just like" an electric circuit (Edison's time), "just like" a computer CPU (21st century), and so on... You're just assuming what you should prove
- mrbungie 2y ago
  He probably didn't need petabytes of reddit posts and millions of gpu-hours to parrot that though. I still don't buy the "we do the same as LLMs" discourse. Of course one could hypothesize the human brain language center may have some similarities to LLMs, but the differences in resource usage and how those resources are used to train humans and LLMs are remarkable and may indicate otherwise.
  - shwouchk 2y ago
    Not text, he had petabytes of video, audio, and other sensory inputs. Heck, a baby sees petabytes of video before first word is spoken And he probably cant quote Shakespeare as well ;)

n144q 2y ago

> GPT-4 would already have qualified as AGI about a decade ago. Did you just make that up?

OtomotO 2y ago
Probably asked an "AI"
wat10000 2y ago
A lot of people held that passing the Turing Test would indicate human-level intelligence. GPT-4 passes.
- bigpingo 2y ago
  Link to GPT-4 passing the turing test? Tried googling, could not find anything.
  - wat10000 2y ago
    Google must be really going downhill. DDG “gpt turing test” provides nothing but relevant links. Here’s a paper: https://arxiv.org/pdf/2405.08007 https://arxiv.org/pdf/2405.08007

OtomotO 2y ago

The last four years? ELIZA 2.0