6 ms·

It's a form of denial. We're getting another "de-thronement of man" on the order of Copernicus and Darwin. Some get excited, others turn away in horror. Negatio

It's a form of denial. We're getting another "de-thronement of man" on the order of Copernicus and Darwin. Some get excited, others turn away in horror. Negation is the outward expression of the desire to keep human intelligence wrapped in its mystical veil. One popular idea is that these systems will asymptotically approximate human intelligence because they're trained on mostly human-written texts. Not only is that untrue, it's also directly contradicted by our experience with previous RL-systems, where they seem to breeze right by human ability without even the slightest hiccup.

iugtmkbdfil834 2mo ago
<< One popular idea is that these systems will asymptotically approximate human intelligence because they're trained on mostly human-written texts. Can you elaborate on this? I am clearly not aware of this line of thinking and the related contradiction.
- defgeneric 2mo ago
  Approximate as a limit and not surpass, where the hidden hope actually seems to be that they don't surpass. There are lots of variations floating around, from simple metaphors like "LLM as librarian that speaks to you," to even Sutton's remark that rather than being a case where we've taken the "bitter lesson" to heart, LLMs may be a yet another case where we're limited by it.
disgruntledphd2 2mo ago
> with previous RL-systems Most human systems are much, much, much more complicated than most closed world games (which is where RL approaches have seen massive success, mostly through self-play). Like LLMs are great, but I honestly can't see us getting actual general intelligence out of them.
- layla5alive 2mo ago
  They already possess general intelligence by many metrics. Sure they miss a few, but that's nitpicky and goalpost shifty - lots of humans make errors of all sorts as well, or incur brain damage limiting them in one or a few areas of intelligence - we don't then say they are not general intelligence anymore. I think maybe you mean superintelligence, which is a more fair critique.
  - disgruntledphd2 2mo ago
    > They already possess general intelligence by many metrics. Can you share the metrics you are using for this assessment? They are really powerful tools, but a quick glance at their thinking tokens (which is a bad name, tbh) rapidly disabuses me of the notion that they are general intelligences. They possess large amounts of crystallised intelligence (i.e. they have a lot of knowledge), but their fluid intelligence is definitely lower than the human median.
    - layla5alive 2mo ago
      My take is that fluid intelligence (quite a bit) below human median is still general intelligence (that is, general intelligence doesn't mean no gaps). It feels like we have collectively goal-post shifted the definition of AGI to be closer to that of ASI. One of those "what do you call a doctor who graduated at the bottom of their class" type things. I think despite their genuine deficits (and there are many), frontier LLMs have basically cleared the minimum AGI bar. Though I realize a lot of people don't agree with this take :)

iugtmkbdfil834 2mo ago

<< One popular idea is that these systems will asymptotically approximate human intelligence because they're trained on mostly human-written texts. Can you elaborate on this? I am clearly not aware of this line of thinking and the related contradiction.

defgeneric 2mo ago
Approximate as a limit and not surpass, where the hidden hope actually seems to be that they don't surpass. There are lots of variations floating around, from simple metaphors like "LLM as librarian that speaks to you," to even Sutton's remark that rather than being a case where we've taken the "bitter lesson" to heart, LLMs may be a yet another case where we're limited by it.

disgruntledphd2 2mo ago

> with previous RL-systems Most human systems are much, much, much more complicated than most closed world games (which is where RL approaches have seen massive success, mostly through self-play). Like LLMs are great, but I honestly can't see us getting actual general intelligence out of them.

layla5alive 2mo ago
They already possess general intelligence by many metrics. Sure they miss a few, but that's nitpicky and goalpost shifty - lots of humans make errors of all sorts as well, or incur brain damage limiting them in one or a few areas of intelligence - we don't then say they are not general intelligence anymore. I think maybe you mean superintelligence, which is a more fair critique.
- disgruntledphd2 2mo ago
  > They already possess general intelligence by many metrics. Can you share the metrics you are using for this assessment? They are really powerful tools, but a quick glance at their thinking tokens (which is a bad name, tbh) rapidly disabuses me of the notion that they are general intelligences. They possess large amounts of crystallised intelligence (i.e. they have a lot of knowledge), but their fluid intelligence is definitely lower than the human median.
  - layla5alive 2mo ago
    My take is that fluid intelligence (quite a bit) below human median is still general intelligence (that is, general intelligence doesn't mean no gaps). It feels like we have collectively goal-post shifted the definition of AGI to be closer to that of ASI. One of those "what do you call a doctor who graduated at the bottom of their class" type things. I think despite their genuine deficits (and there are many), frontier LLMs have basically cleared the minimum AGI bar. Though I realize a lot of people don't agree with this take :)