6 ms·

> if we transcribe your brain into a simulation and give it a tickrate, you will be just autocomplete too. The underlying mechanisms for producing language are

> if we transcribe your brain into a simulation and give it a tickrate, you will be just autocomplete too. The underlying mechanisms for producing language are very different, in the same way birds and airplanes have different mechanisms for flying.

CamperBob2 4d ago
No, not really. If the old "birds and airplanes" or "swimmers and submarines" metaphors were valid, then language alone wouldn't be enough to encode and embody reasoning capabilities, as has recently become evident. LLMs are more like human minds than we are willing to admit. Such reluctance is perhaps the least surprising aspect of any of this.
- overfeed 4d ago
  > then language alone wouldn't be enough to encode and embody reasoning capabilities I disagree, language is the product of, not the mechanism for thought. People who lose their faculty of speech (or haven't gained them) have complete thoughts and executive function. CoT is a hack to use language (i.e. autoregression) to simulate reasoning, and it's very effective at it. Human minds can acquire, hold and use axioms as building blocks for actual reasoning, LLMs use statistical likelihood.
  - stratos123 4d ago
    > Human minds can acquire, hold and use axioms as building blocks for actual reasoning, LLMs use statistical likelihood. This is not even slightly true. Even when trying, humans commit logical mistakes at notable rates, because the behavior of our minds is inherently nondeterministic. Human minds are not built for logic and need to twist themselves into knots and rely on symbolic representations to do it. There is such a thing as valid reasoning - predicate calculus, decision theory - and humans only emulate it with some accuracy, in hacky ways, and only because our brains were imperfectly taught to do that over millenia of evolutionary pressure. LLMs are much the same way.
    - godelski 4d ago
      > humans commit logical mistakes at notable rates You're talking past the parent's claim. If your axioms are wrong then of course your logic will be wrong. But then again, maybe to support your point people frequently say "start from first principles" when those are usually the thing that needs to be found, not the place you start. But LLMs, like humans, love to be confident about things they aren't sufficiently trained on
      - HappMacDonald 3d ago
        > If your axioms are wrong then of course your logic will be wrong. Equally, even if your axioms are right your logical execution can still fail. Neither individual humans nor LLMs get either of those steps right as a default. Just as LLMs must use hacky CoT and other systems of error checking and correcting to even begin to emulate reasoning, humans must do the same thing. We are apes. We are driven by emotion and solve problems either when we are forced to or as a sport. We did create the ultimate automata of logical execution: the deterministic von Neumann Machine, but a human brain in no way reflects that operation. Nor, as you have observed, does an LLM despite using one as its substrate.
  - teravor 4d ago
    when an LLM is trained backpropagation reaches into the entirety of the key value table. or rather, all the layers (not just the tokens) which come before the current tick. langauge is only produced by the final layer every tick. and every layer can only interact with other layers on the same level. the mechanics of LLMs and the restriction in how we can train them makes it appear as though all we are doing is forcing language onto them but once RL gets involved all bets are off regarding what's happening inside them (it's quite possible that a static corpus alone is sufficient for all the bets being off).
- godelski 4d ago
  FYI, math is a language. Which is actually an important part of the Navier-Stokes conversation. Solving hard problems expands the vocabulary. The problems are hard, illustrating a region we know where the language is insufficient. So the point isn't so much to solve the specific problem, but to figure out how to discuss problems like it. But there's a big difference between talking about something in an extremely convoluted manner then people struggle to understand and inventing a new word that simplifies our discussions. Though this is grossly oversimplified. It's a HN comment, not a lecture on metamathematics or metaml
- joquarky 4d ago
  And why wouldn't nature take advantage of such a relatively simple and effective pattern, given all of the emergent behavior it produces, plus the ability to adapt? Human egos are probably the reason we also resisted the concept of heliocentrism.

CamperBob2 4d ago

No, not really. If the old "birds and airplanes" or "swimmers and submarines" metaphors were valid, then language alone wouldn't be enough to encode and embody reasoning capabilities, as has recently become evident. LLMs are more like human minds than we are willing to admit. Such reluctance is perhaps the least surprising aspect of any of this.

overfeed 4d ago
> then language alone wouldn't be enough to encode and embody reasoning capabilities I disagree, language is the product of, not the mechanism for thought. People who lose their faculty of speech (or haven't gained them) have complete thoughts and executive function. CoT is a hack to use language (i.e. autoregression) to simulate reasoning, and it's very effective at it. Human minds can acquire, hold and use axioms as building blocks for actual reasoning, LLMs use statistical likelihood.
- stratos123 4d ago
  > Human minds can acquire, hold and use axioms as building blocks for actual reasoning, LLMs use statistical likelihood. This is not even slightly true. Even when trying, humans commit logical mistakes at notable rates, because the behavior of our minds is inherently nondeterministic. Human minds are not built for logic and need to twist themselves into knots and rely on symbolic representations to do it. There is such a thing as valid reasoning - predicate calculus, decision theory - and humans only emulate it with some accuracy, in hacky ways, and only because our brains were imperfectly taught to do that over millenia of evolutionary pressure. LLMs are much the same way.
  - godelski 4d ago
    > humans commit logical mistakes at notable rates You're talking past the parent's claim. If your axioms are wrong then of course your logic will be wrong. But then again, maybe to support your point people frequently say "start from first principles" when those are usually the thing that needs to be found, not the place you start. But LLMs, like humans, love to be confident about things they aren't sufficiently trained on
    - HappMacDonald 3d ago
      > If your axioms are wrong then of course your logic will be wrong. Equally, even if your axioms are right your logical execution can still fail. Neither individual humans nor LLMs get either of those steps right as a default. Just as LLMs must use hacky CoT and other systems of error checking and correcting to even begin to emulate reasoning, humans must do the same thing. We are apes. We are driven by emotion and solve problems either when we are forced to or as a sport. We did create the ultimate automata of logical execution: the deterministic von Neumann Machine, but a human brain in no way reflects that operation. Nor, as you have observed, does an LLM despite using one as its substrate.
- teravor 4d ago
  when an LLM is trained backpropagation reaches into the entirety of the key value table. or rather, all the layers (not just the tokens) which come before the current tick. langauge is only produced by the final layer every tick. and every layer can only interact with other layers on the same level. the mechanics of LLMs and the restriction in how we can train them makes it appear as though all we are doing is forcing language onto them but once RL gets involved all bets are off regarding what's happening inside them (it's quite possible that a static corpus alone is sufficient for all the bets being off).
godelski 4d ago
FYI, math is a language. Which is actually an important part of the Navier-Stokes conversation. Solving hard problems expands the vocabulary. The problems are hard, illustrating a region we know where the language is insufficient. So the point isn't so much to solve the specific problem, but to figure out how to discuss problems like it. But there's a big difference between talking about something in an extremely convoluted manner then people struggle to understand and inventing a new word that simplifies our discussions. Though this is grossly oversimplified. It's a HN comment, not a lecture on metamathematics or metaml
joquarky 4d ago
And why wouldn't nature take advantage of such a relatively simple and effective pattern, given all of the emergent behavior it produces, plus the ability to adapt? Human egos are probably the reason we also resisted the concept of heliocentrism.