7 ms·

Whether Bender intended it or not, the term has an inherently pejorative sense. "Parroting" is not really indicative of what modern LLMs do. However, when most

Whether Bender intended it or not, the term has an inherently pejorative sense. "Parroting" is not really indicative of what modern LLMs do. However, when most people bring it up as a criticism of "AI in general" in 2026, they're using it as a pointer to all of the social/environmental criticisms, rather than the technological capabilities.

delis-thumbs-7e 2mo ago
Why is it not indicative of what LLM’s do?
- TaupeRanger 2mo ago
  Really? Are you under the impression that parrots are able to synthesize their input and create entirely new, useful outputs which they have never heard before?
  - delis-thumbs-7e 2mo ago
    Yes, I am: https://en.wikipedia.org/wiki/Alex_(parrot) https://en.wikipedia.org/wiki/Alex_(parrot)
  - softwaredoug 2mo ago
    The "stochastic" is the key modifier to "parrot". > "Stochastic" means having a random pattern or variable that can be analyzed statistically but not predicted precisely
- naasking 2mo ago
  Because they don't just parrot, they interpolate, which is why they have such varied abilities. You can't explain the range of behaviours they have with just parroting, and once you accept that, why shouldn't this qualify as some form of intelligence?
  - delis-thumbs-7e 2mo ago
    There’s a lot of confusion here on what parrots exactly are capable of doing and assumptions how smart they are. I also don’t quite see what you mean by behaviour in respect to LLM’s, possibly “agentic” tool usage? Because I’m fairly certain you can explain their behaviour as “stochastic parroting” combined with very strong program, ie. harness, to interact with other systems. And perhaps this all is “some kind of intelligence”, but then you have to be very careful with what you mean by intelligence. It becomes a terribly slippery surface described like this and you see, who knows where this gradient descent takes us if we are not careful. So personally I would still rather undersell it. But parrot or not, it is still a terribly useful little bird indeed.
koverstreet 2mo ago
Yeah, there's some beautiful math underlying what LLMs are doing, and it's the same math our neocortex runs on.
- ainch 2mo ago
  Could you provide more detail? My understanding is that the neocortex is predominantly focused on forwards simulation, which seems distinct to how transformers operate.
  - koverstreet 2mo ago
    That's fundamental to how anything that compresses/understands the world has to work, in the Kolmogeravian sense. That's why people denigrate LLMs as being just "next token predictors" - they're not wrong, but they're missing the point. Because to do that kind of prediction out in the world you have to build up an accurate model of reality - a model that includes yourself! Which is why we and LLMs are self aware. For the "how", it's been known for some time that LLMs operate on a Reimannian manifold - the semantic manifold - and that's a good place to start if you want to learn how they actually work; how a Reimannian manifold (plus some extra structure on top) can represent natural language in a form you can do work with is the part I find particularly beautiful. At a high level, the neocortex and LLMs appear to compute on the manifold in basically the same way - though a lot of the details are different; both are more sophisticated in some areas and less in others.
    - ainch 2mo ago
      I'm afraid the precise connection you're making isn't totally obvious to me. As far as prediction - I mean sure the cortex and LLMs do prediction, but then so can RNNs or diffusion models or any other generative model. Really any ML architecture is learning to compress its environment in pursuit of modelling. More broadly, the predictive brain model would suggest that all of the brain, not just the neocortex, is dedicated to prediction. What would you say makes LLMs similar to the neocortex, rather than the basal ganglia or Broca's area? Similarly, if you agree with the Manifold Hypothesis, then all machine learning models operate on manifolds. I agree it's an exciting thought, but then I don't know what would distinguish an LLM from a VAE or SVM in terms of operating over a low-dimensional manifold embedded in high dimensional spaces - maybe just scale?
inspectorSlap 2mo ago
I think stochastic parroting is really a very accurate description of what they do (if underserving of the overall usefulness of LLMs). As long as you consider they are parroting from the whole of human intelligence. Its just that as they have gotten more sophisticated, the amount of gates, guardrails, and tertiary tools add variety. Trace any LLM hallucination back to provenance and you begin to see how the stochastic parrot works.
tptacek 2mo ago
We're all going to stay fixated on what exactly a "parrot" is, but the paper doesn't rest on the definition of tropical birds: it advances a specific and detailed hypothesis about how language models don't operate with communicative intent, and in fact the meaning humans obtain from LLM output is a cognitive pattern matching illusion. Those claims have not held up at all. At this point, I think the authors are really counting on people not to read their paper.
thomastjeffery 2mo ago
How can you articulate a criticism without a repetition of your criticism becoming "inherently pejorative"? You can't. That's just the way the news goes. Essentially, what you are saying is that because some people somewhere frame a statement as pejorative, the statement itself is inherently pejorative. By that logic, every criticism ever articulated is inherently an insult.

delis-thumbs-7e 2mo ago

Why is it not indicative of what LLM’s do?

TaupeRanger 2mo ago
Really? Are you under the impression that parrots are able to synthesize their input and create entirely new, useful outputs which they have never heard before?
- delis-thumbs-7e 2mo ago
  Yes, I am: https://en.wikipedia.org/wiki/Alex_(parrot) https://en.wikipedia.org/wiki/Alex_(parrot)
- softwaredoug 2mo ago
  The "stochastic" is the key modifier to "parrot". > "Stochastic" means having a random pattern or variable that can be analyzed statistically but not predicted precisely
naasking 2mo ago
Because they don't just parrot, they interpolate, which is why they have such varied abilities. You can't explain the range of behaviours they have with just parroting, and once you accept that, why shouldn't this qualify as some form of intelligence?
- delis-thumbs-7e 2mo ago
  There’s a lot of confusion here on what parrots exactly are capable of doing and assumptions how smart they are. I also don’t quite see what you mean by behaviour in respect to LLM’s, possibly “agentic” tool usage? Because I’m fairly certain you can explain their behaviour as “stochastic parroting” combined with very strong program, ie. harness, to interact with other systems. And perhaps this all is “some kind of intelligence”, but then you have to be very careful with what you mean by intelligence. It becomes a terribly slippery surface described like this and you see, who knows where this gradient descent takes us if we are not careful. So personally I would still rather undersell it. But parrot or not, it is still a terribly useful little bird indeed.

koverstreet 2mo ago

Yeah, there's some beautiful math underlying what LLMs are doing, and it's the same math our neocortex runs on.

ainch 2mo ago
Could you provide more detail? My understanding is that the neocortex is predominantly focused on forwards simulation, which seems distinct to how transformers operate.
- koverstreet 2mo ago
  That's fundamental to how anything that compresses/understands the world has to work, in the Kolmogeravian sense. That's why people denigrate LLMs as being just "next token predictors" - they're not wrong, but they're missing the point. Because to do that kind of prediction out in the world you have to build up an accurate model of reality - a model that includes yourself! Which is why we and LLMs are self aware. For the "how", it's been known for some time that LLMs operate on a Reimannian manifold - the semantic manifold - and that's a good place to start if you want to learn how they actually work; how a Reimannian manifold (plus some extra structure on top) can represent natural language in a form you can do work with is the part I find particularly beautiful. At a high level, the neocortex and LLMs appear to compute on the manifold in basically the same way - though a lot of the details are different; both are more sophisticated in some areas and less in others.
  - ainch 2mo ago
    I'm afraid the precise connection you're making isn't totally obvious to me. As far as prediction - I mean sure the cortex and LLMs do prediction, but then so can RNNs or diffusion models or any other generative model. Really any ML architecture is learning to compress its environment in pursuit of modelling. More broadly, the predictive brain model would suggest that all of the brain, not just the neocortex, is dedicated to prediction. What would you say makes LLMs similar to the neocortex, rather than the basal ganglia or Broca's area? Similarly, if you agree with the Manifold Hypothesis, then all machine learning models operate on manifolds. I agree it's an exciting thought, but then I don't know what would distinguish an LLM from a VAE or SVM in terms of operating over a low-dimensional manifold embedded in high dimensional spaces - maybe just scale?

inspectorSlap 2mo ago

I think stochastic parroting is really a very accurate description of what they do (if underserving of the overall usefulness of LLMs). As long as you consider they are parroting from the whole of human intelligence. Its just that as they have gotten more sophisticated, the amount of gates, guardrails, and tertiary tools add variety. Trace any LLM hallucination back to provenance and you begin to see how the stochastic parrot works.

tptacek 2mo ago

We're all going to stay fixated on what exactly a "parrot" is, but the paper doesn't rest on the definition of tropical birds: it advances a specific and detailed hypothesis about how language models don't operate with communicative intent, and in fact the meaning humans obtain from LLM output is a cognitive pattern matching illusion. Those claims have not held up at all. At this point, I think the authors are really counting on people not to read their paper.

thomastjeffery 2mo ago

How can you articulate a criticism without a repetition of your criticism becoming "inherently pejorative"? You can't. That's just the way the news goes. Essentially, what you are saying is that because some people somewhere frame a statement as pejorative, the statement itself is inherently pejorative. By that logic, every criticism ever articulated is inherently an insult.