8 ms·

I suspect AGI will be one of those things that you can't describe it exactly, but you'll know it when you see it.

by jjeaff 1y ago

ninetyninenine 1y ago
I suspect everyone will call it a stochastic parrot because it did this one thing not right. And this will continue into the far far future even when it becomes sentient we will completely miss it.
- Jensson 1y ago
  Once it pushed out most humans from white collar labor so the remaining humans work in blue collar jobs people wont say its just a stochastic parrot.
  - myk9001 1y ago
    Maybe, maybe not. Power loom pushed a lot of humans out of the textile factory jobs, yet noone claims power loom is the AGI.
    - Jensson 1y ago
      Not a lot, I mean basically everyone, to the point where most companies doesn't need to pay humans to think anymore.
      - myk9001 1y ago
        Well, I'm too lazy to look up how many weavers were displaced back then and that's why I said a lot. Maybe all of them, since they weren't trained to operate the new machines. Anyway, sorry for a digression, my point is LLM replacing white collar workers doesn't necessarily imply it's generally intelligent -- it may but doesn't have to be. Although if it gets to a point where companies are running dark office buildings (by analogy with dark factories) -- yes, it's AGI by then.
  - jimbokun 1y ago
    Or become shocked to realize humans are basically statistical parrots too.
  - butlike 1y ago
    The blue collar jobs are more entertaining anyways, provided you take the monetary inequality away.
    - int_19h 1y ago
      Tastes differ.
- AstralStorm 1y ago
  It's more than that but less than intelligence. Its generalization capabilities are a bit on the low side, and memory is relatively bad. But it is much more than just a parrot now, it can handle some of basic logic, but not follow given patterns correctly for novel problems. I'd liken it to something like a bird, extremely good at specialized tasks but failing a lot of common ones unless repeatedly shown the solution. It's not a corvid or a parrot yet. Fails rather badly at detour tests. It might be sentient already though. Someone needs to run a test if it can discern itself and another instance of itself in its own work.
  - Jensson 1y ago
    > It might be sentient already though. Someone needs to run a test if it can discern itself and another instance of itself in its own work. It doesn't have any memory, how could it tell itself from a clone of itself?
    - AstralStorm 1y ago
      Similarity match. For that you need to understand reflexively how you think and write. It's a fun test to give a person something they have written but do not remember. Most people can still spot it. It's easier with images though. Especially a mirror. For DallE, the test would be if it can discern its own work from human generated image. Especially of you give it an imaginative task like drawing a representation of itself.
    - butlike 1y ago
      It doesn't have any memory _you're aware of_. A semiconductor can hold state, so it has memory.
    - ben_w 1y ago
      People already share viral clips of AI recognising other AI, but I've not seen real scientific study of if this is due to a literary form of passing a mirror test, or if it's related to the way most models openly tell everyone they talk to that they're an AI. As for "how", note that memory isn't one single thing even in humans: https://en.wikipedia.org/wiki/Memory https://en.wikipedia.org/wiki/Memory I don't want to say any of these are exactly equivalent to any given aspect of human memory, but I would suggest that LLMs behave kinda like they have: (1) Sensory memory in the form of a context window — and in this sense are wildly superhuman because for a human that's about one second, whereas an AI's context window is about as much text as a human goes through in a week (actually less because we don't only read, other sensory modalities do matter; but for scale: equivalent to what you read in a week) (2) Short term memory in the form of attention heads — and in this sense are wildly superhuman, because humans pay attention to only 4–5 items whereas DeepSeek v3 defaults to 128. (3) The training and fine-tuning process itself that allows these models to learn how to communicate with us. Not sure what that would count as. Learned skill? Operant conditioning? Long term memory? It can clearly pick up different writing styles, because it can be made to controllably output in different styles — but that's an "in principle" answer. None of Claude 3.7, o4-mini, DeepSeek r1, could actually identify the authorship of a (n=1) test passage I asked 4o to generate for me.
NitpickLawyer 1y ago
> but you'll know it when you see it. I agree, but with the caveat that it's getting harder and harder with all the hype / doom cycles and all the goalpost moving that's happening in this space. IMO if you took gemini2.5 / claude / o3 and showed it to people from ten / twenty years ago, they'd say that it is unmistakably AGI.
- Jensson 1y ago
  > IMO if you took gemini2.5 / claude / o3 and showed it to people from ten / twenty years ago, they'd say that it is unmistakably AGI. No they wouldn't, since those still can't replace human white collar workers even at many very basic tasks. Once AGI is here most white collar jobs are gone, you'd only need to hire geniuses at most.
  - zaptrem 1y ago
    Which part of "General Intelligence" requires replacing white collar workers? A middle schooler has general intelligence (they know about and can do a lot of things across a lot of different areas) but they likely can't replace white collar workers either. IMO GPT-3 was AGI, just a pretty crappy one.
    - Jensson 1y ago
      > A middle schooler has general intelligence (they know about and can do a lot of things across a lot of different areas) but they likely can't replace white collar workers either. Middle schoolers replace white collars workers all the time, it takes 10 years for them to do it but they can do it. No current model can do the same since they aren't able to learn over time like a middle schooler.
      - sebastiennight 1y ago
        Compared to someone who graduated middle school on November 30th, 2022 (2.5 years ago, would you say that today's gemini 2.5 pro has NOT gained intelligence faster? I mean, if you're a CEO or middle manager and you have the choice of hiring this middle schooler for general office work, or today's gemini-2.5-pro, are you 100% saying the ex-middle-schooler is definitely going to give you best bang for your buck? Assuming you can either pay them $100k a year, or spend the $100k on gemini inference.
afro88 1y ago
This is part of what the article is about
torginus 1y ago
I still can't have an earnest conversation or bounce ideas off of any LLM - all of them seem to be a cross between a sentient encyclopedia and a constraint solver. They might get more powerful but I feel like they're still missing something.
- itchyjunk 1y ago
  Why are you not able to have an earnest conversation with an LLM? What kind of ideas are you not able to bounce of LLMs? These seem to be the type of use cases where LLMs have generally shined for me.
  - 9dev 1y ago
    Eh, I am torn on this. I had some great conversations on random questions or conceptual ideas, but also some where the models instructions shone through way too clearly. Like, when you ask something like "I’m working on the architecture of this system, can you let me know what you think and if there’s anything obvious to improve on?"—the model will always a) flatter me for my amazing concept, b) point out the especially laudable parts of it, and c) name a few obvious but not-really-relevant parts (e.g. "always be careful with secrets and passwords"). However, it will not actually point out higher level design improvements, or alternative solutions. It’s always just regurgitating what I’ve told it about. That is semi-useful, most of the time.
    - john_minsk 1y ago
      Because it spits out the most probable answer, which is based on endless copycat articles online written by marketers for C-level decision makers to sell their software. AI doesn't go and read a book on best practices, then comes back saying "Now I know Kung Fu of Software Implementation" and then critically thinks looking at your plan step by step and provides answer. These systems, for now, don't work like that. Would you disagree?
      - 9dev 1y ago
        How come we’re discussing if they’re artificial general intelligence then?
jimbokun 1y ago
We have all seen it and are now just in severe denial.
dgs_sgd 1y ago
This is actually how a supreme court justice defined the test for obscenity. > The phrase "I know it when I see it" was used in 1964 by United States Supreme Court Justice Potter Stewart to describe his threshold test for obscenity in Jacobellis v. Ohio
- sweetjuly 1y ago
  The reason why it's so famous though (and why some people tend to use it in a tongue in cheek manner) is because "you know it when you see it" is a hilariously unhelpful and capricious threshold, especially when coming from the Supreme Court. For rights which are so vital to the fabric of the country, the Supreme Court recommending we hinge free speech on—essentially—unquantifiable vibes is equal parts bizarre and out of character.
DesiLurker 1y ago
my 2c on this is that if you interact with any current llm enough you can mentally 'place' its behavior and responses. when we truly have AGI+/ASI my guess is that it will be like that old adage of blind men feeling & describing an elephant for the first time. we just wont be able to fully understand its responses. it would always be something left hanging and then eventually we'll just stop trying. that would be time when the exponential improvement really kicks in. it should suffice to say we are nowhere near that and I dont even believe LLMs are the right architecture for that.

I suspect AGI will be one of those things that you can't describe it exactly, but you'll know it when you see it.

by jjeaff 1y ago

I suspect AGI will be one of those things that you can't describe it exactly, but you'll know it when you see it.

I suspect everyone will call it a stochastic parrot because it did this one thing not right. And this will continue into the far far future even when it becomes sentient we will completely miss it.

Jensson 1y ago
Once it pushed out most humans from white collar labor so the remaining humans work in blue collar jobs people wont say its just a stochastic parrot.
- myk9001 1y ago
  Maybe, maybe not. Power loom pushed a lot of humans out of the textile factory jobs, yet noone claims power loom is the AGI.
  - Jensson 1y ago
    Not a lot, I mean basically everyone, to the point where most companies doesn't need to pay humans to think anymore.
    - myk9001 1y ago
      Well, I'm too lazy to look up how many weavers were displaced back then and that's why I said a lot. Maybe all of them, since they weren't trained to operate the new machines. Anyway, sorry for a digression, my point is LLM replacing white collar workers doesn't necessarily imply it's generally intelligent -- it may but doesn't have to be. Although if it gets to a point where companies are running dark office buildings (by analogy with dark factories) -- yes, it's AGI by then.
- jimbokun 1y ago
  Or become shocked to realize humans are basically statistical parrots too.
- butlike 1y ago
  The blue collar jobs are more entertaining anyways, provided you take the monetary inequality away.
  - int_19h 1y ago
    Tastes differ.
AstralStorm 1y ago
It's more than that but less than intelligence. Its generalization capabilities are a bit on the low side, and memory is relatively bad. But it is much more than just a parrot now, it can handle some of basic logic, but not follow given patterns correctly for novel problems. I'd liken it to something like a bird, extremely good at specialized tasks but failing a lot of common ones unless repeatedly shown the solution. It's not a corvid or a parrot yet. Fails rather badly at detour tests. It might be sentient already though. Someone needs to run a test if it can discern itself and another instance of itself in its own work.
- Jensson 1y ago
  > It might be sentient already though. Someone needs to run a test if it can discern itself and another instance of itself in its own work. It doesn't have any memory, how could it tell itself from a clone of itself?
  - AstralStorm 1y ago
    Similarity match. For that you need to understand reflexively how you think and write. It's a fun test to give a person something they have written but do not remember. Most people can still spot it. It's easier with images though. Especially a mirror. For DallE, the test would be if it can discern its own work from human generated image. Especially of you give it an imaginative task like drawing a representation of itself.
  - butlike 1y ago
    It doesn't have any memory _you're aware of_. A semiconductor can hold state, so it has memory.
  - ben_w 1y ago
    People already share viral clips of AI recognising other AI, but I've not seen real scientific study of if this is due to a literary form of passing a mirror test, or if it's related to the way most models openly tell everyone they talk to that they're an AI. As for "how", note that memory isn't one single thing even in humans: https://en.wikipedia.org/wiki/Memory https://en.wikipedia.org/wiki/Memory I don't want to say any of these are exactly equivalent to any given aspect of human memory, but I would suggest that LLMs behave kinda like they have: (1) Sensory memory in the form of a context window — and in this sense are wildly superhuman because for a human that's about one second, whereas an AI's context window is about as much text as a human goes through in a week (actually less because we don't only read, other sensory modalities do matter; but for scale: equivalent to what you read in a week) (2) Short term memory in the form of attention heads — and in this sense are wildly superhuman, because humans pay attention to only 4–5 items whereas DeepSeek v3 defaults to 128. (3) The training and fine-tuning process itself that allows these models to learn how to communicate with us. Not sure what that would count as. Learned skill? Operant conditioning? Long term memory? It can clearly pick up different writing styles, because it can be made to controllably output in different styles — but that's an "in principle" answer. None of Claude 3.7, o4-mini, DeepSeek r1, could actually identify the authorship of a (n=1) test passage I asked 4o to generate for me.

NitpickLawyer 1y ago

> but you'll know it when you see it. I agree, but with the caveat that it's getting harder and harder with all the hype / doom cycles and all the goalpost moving that's happening in this space. IMO if you took gemini2.5 / claude / o3 and showed it to people from ten / twenty years ago, they'd say that it is unmistakably AGI.

Jensson 1y ago
> IMO if you took gemini2.5 / claude / o3 and showed it to people from ten / twenty years ago, they'd say that it is unmistakably AGI. No they wouldn't, since those still can't replace human white collar workers even at many very basic tasks. Once AGI is here most white collar jobs are gone, you'd only need to hire geniuses at most.
- zaptrem 1y ago
  Which part of "General Intelligence" requires replacing white collar workers? A middle schooler has general intelligence (they know about and can do a lot of things across a lot of different areas) but they likely can't replace white collar workers either. IMO GPT-3 was AGI, just a pretty crappy one.
  - Jensson 1y ago
    > A middle schooler has general intelligence (they know about and can do a lot of things across a lot of different areas) but they likely can't replace white collar workers either. Middle schoolers replace white collars workers all the time, it takes 10 years for them to do it but they can do it. No current model can do the same since they aren't able to learn over time like a middle schooler.
    - sebastiennight 1y ago
      Compared to someone who graduated middle school on November 30th, 2022 (2.5 years ago, would you say that today's gemini 2.5 pro has NOT gained intelligence faster? I mean, if you're a CEO or middle manager and you have the choice of hiring this middle schooler for general office work, or today's gemini-2.5-pro, are you 100% saying the ex-middle-schooler is definitely going to give you best bang for your buck? Assuming you can either pay them $100k a year, or spend the $100k on gemini inference.

afro88 1y ago

This is part of what the article is about

torginus 1y ago

I still can't have an earnest conversation or bounce ideas off of any LLM - all of them seem to be a cross between a sentient encyclopedia and a constraint solver. They might get more powerful but I feel like they're still missing something.

itchyjunk 1y ago
Why are you not able to have an earnest conversation with an LLM? What kind of ideas are you not able to bounce of LLMs? These seem to be the type of use cases where LLMs have generally shined for me.
- 9dev 1y ago
  Eh, I am torn on this. I had some great conversations on random questions or conceptual ideas, but also some where the models instructions shone through way too clearly. Like, when you ask something like "I’m working on the architecture of this system, can you let me know what you think and if there’s anything obvious to improve on?"—the model will always a) flatter me for my amazing concept, b) point out the especially laudable parts of it, and c) name a few obvious but not-really-relevant parts (e.g. "always be careful with secrets and passwords"). However, it will not actually point out higher level design improvements, or alternative solutions. It’s always just regurgitating what I’ve told it about. That is semi-useful, most of the time.
  - john_minsk 1y ago
    Because it spits out the most probable answer, which is based on endless copycat articles online written by marketers for C-level decision makers to sell their software. AI doesn't go and read a book on best practices, then comes back saying "Now I know Kung Fu of Software Implementation" and then critically thinks looking at your plan step by step and provides answer. These systems, for now, don't work like that. Would you disagree?
    - 9dev 1y ago
      How come we’re discussing if they’re artificial general intelligence then?

jimbokun 1y ago

We have all seen it and are now just in severe denial.

dgs_sgd 1y ago

This is actually how a supreme court justice defined the test for obscenity. > The phrase "I know it when I see it" was used in 1964 by United States Supreme Court Justice Potter Stewart to describe his threshold test for obscenity in Jacobellis v. Ohio

sweetjuly 1y ago
The reason why it's so famous though (and why some people tend to use it in a tongue in cheek manner) is because "you know it when you see it" is a hilariously unhelpful and capricious threshold, especially when coming from the Supreme Court. For rights which are so vital to the fabric of the country, the Supreme Court recommending we hinge free speech on—essentially—unquantifiable vibes is equal parts bizarre and out of character.

DesiLurker 1y ago

my 2c on this is that if you interact with any current llm enough you can mentally 'place' its behavior and responses. when we truly have AGI+/ASI my guess is that it will be like that old adage of blind men feeling & describing an elephant for the first time. we just wont be able to fully understand its responses. it would always be something left hanging and then eventually we'll just stop trying. that would be time when the exponential improvement really kicks in. it should suffice to say we are nowhere near that and I dont even believe LLMs are the right architecture for that.