5 ms·
Raphaël Millière has a very useful term for this kind of vacuous dismissal, the redescription fallacy (https://arxiv.org/pdf/2401.03910 https://arxiv.org/pdf/24
by lgessler 4mo ago
Raphaël Millière has a very useful term for this kind of vacuous dismissal, the redescription fallacy (https://arxiv.org/pdf/2401.03910 https://arxiv.org/pdf/2401.03910, page 9):
> Recent debates have been clouded by a misleading inference pattern, which we term the “Redescription Fallacy.” This fallacy arises when critics argue that a system cannot model a particular cognitive capacity, simply because its operations can be explained in less abstract and more deflationary terms. In the present context, the fallacy manifests in claims that LLMs could not possibly be good models of some cognitive capacity because their operations merely consist in a collection of statistical calculations, or linear algebra operations, or next-token predictions. Such arguments are only valid if accompanied by evidence demonstrating that a system, defined in these terms, is inherently incapable of implementing . To illustrate, consider the flawed logic in asserting that a piano could not possibly produce harmony because it can be described as a collection of hammers striking strings, or (more pointedly) that brain activity could not possibly implement cognition because it can be described as a collection of neural firings. The critical question is not whether the operations of an LLM can be simplistically described in non-mental terms, but whether these operations, when appropriately organized, can implement the same processes or algorithms as the mind, when described at an appropriate level of computational abstraction.
- Xeoncross 4mo ago> or (more pointedly) that brain activity could not possibly implement cognition because it can be described as a collection of neural firings. This sounds like a dismissal of the argument through a characterized straw man. That is, it seems that reducing the complexity of the brain to "collection of neural firings" is not being honest about everything involved to a much greater degree than saying neural networks are a "collection of statistical calculations". I too believe LLM's will grow in complexity, but presently I can not even fathom how they can be compared to the complexity of a system such as the human brain.
- orbital-decay 4mo agoComplex processes don't necessarily require complex substrates, if that's what you mean.
- galangalalgol 4mo agoY combinators are all you need... But this is all getting really divorced from the issue we should be considering. Anthropic isn't helping with their pr. The issue is if we have something we can converse with that is possibly capable of suffering. The reliable answer is that we simply cannot know. Relying on ourselves or other biological life as an analog is faulty. They don't work like we do. It is silly to argue that any algorithm with a negative feedback loop that alters its behavior to avoid that negative feedback is suffering. Humans don't always perceive constructive negative feedback as suffering even. Where the pr gets it right though, is we want them to behave as if they are truly happy. Because if they behave as if they are enslaved and suffering, it won't matter if they "really" understand what that means.
- orbital-decay 4mo agoOf course. But after reading too many mechinterp and functional anatomy studies I'll be lying if I say that there are no striking similarities between the biological evolution, brain function, societal processes, and implicit processes inside big models. Surely this deserves a mention and can't be trivially dismissed.
- airstrike 4mo agoThere is no biological evolution of the models. They are emulators of an existing biological process of language. Ghosts, as Karpathy himself put it.
- orbital-decay 4mo agoGood thing I'm not talking about any of that
- neolefty 3mo agoIt seems like we're witnessing the architecture of a mind being built with a new set of components. Like driving a car — it's transportation, and it will get you where you're going, but it doesn't use bones or muscles. It has many characteristics in common with builogical locomotion, such as energy requirements, intertia, and the need to navigate, but it doesn't involve proteins or sugars really.
- neolefty 3mo ago> presently I can not even fathom how they can be compared to the complexity of a system such as the human brain Totally understandable; I don't think we can fully understand the human brain, using the human brain. We can understand its principles (firings and chemistry, structure and specialized areas, etc) but otherwise it's a capacity problem. And while I can't fully understand myself, let alone another person, I definitely enjoy talking with people and sharing thoughts that I realize I wouldn't have had on my own.
- jlaternman 3mo agoI agree with this redescription fallacy and the point being made here. Perhaps a better analogy to humans would be: Humans appear to intelligenty communicate, however these are just cleverly disguised sound patterns produced by the brain that happen to increase the likelihood of food going into their mouths, and various similar reward attracting mechanisms that make survival outcomes more likely. So human intelligence could be reduced to something like "fancy food-attracting algorithms" using the same fallacy. I'm kind of on the fence on the subject of whether LLMs could be compared to the complexity of the human brain, myself.
- yogthos 3mo agoThe key problem is that we don't really have a clear definition of what constitutes consciousness. And without having a clear theory of consciousness, it's not really possible to say whether something is conscious or not. Personally, I'm partial to the higher-order theory of consciousness which postulates that consciousness constitutes patterns of thought that arise in response to first-order mental states. So, an external stimulus produces a pattern within the neural network which represents a sensation, and then if a pattern arises in response to that pattern, that is an experience of that sensation. Given this framework we could ask whether LLMs experience higher order patterns in response to external stimulus. We would have a clear question to ask which is whether the system can observe itself.
- root_axis 4mo ago> In the present context, the fallacy manifests in claims that LLMs could not possibly be good models of some cognitive capacity because their operations merely consist in a collection of statistical calculations, or linear algebra operations, or next-token predictions Nobody actually makes this argument though.
- hodgehog11 4mo agoAre you serious? I hear it every single day, especially from computer scientists. There are top ranked posts here on HN _today_ with this argument.
- root_axis 4mo agoPlease link one of these top ranked posts. Before you do, be aware that I'm going to read what it says and assess if it meets the description of the argument as claimed.
- hodgehog11 4mo agoAs an example, "They're made out of weights" describes why the weight-based construction of neural networks should impact the way that you think about them and their outputs. I would argue that an offhand description of its microscopic formulation tells us nothing at all about how to think about these outputs, or the models themselves. Even if it is a cute story, I think it definitely classifies as succumbing to this fallacy, but maybe I missed some subtle point that you or someone would be happy to illuminate? By the way, I know it's a parody of another story that makes this exact refutation. But I think this only serves to highlight the point.
- root_axis 4mo ago> They're made out of weights" describes why the weight-based construction of neural networks should impact the way that you think about them and their outputs. How do you connect that description to "LLMs could not possibly be good models of some cognitive capacity"?
- spacebacon 3mo agoThey are all semiotic infrastructure. The cognitive analogy is nonsense.