5 ms·

too much thinking https://gist.github.com/gavi/b9985f730f5deefe49b6a28e5569d467 https://gist.github.com/gavi/b9985f730f5deefe49b6a28e5569d46...

• fzzzy 1y ago
  My impression from running the first R1 release locally was that it also does too much thinking.
  • cluckindan 1y ago
    It does not do any thinking. It is a statistical model, just like the rest of them.
    • robmccoll 1y ago
      What are we doing when we think?
      • otabdeveloper4 1y ago
        We don't know yet. But we do know it's certainly not statistical token prediction. (People can do statistical token prediction too, but that's called "bullshitting", not "thinking". Thinking is a much wider class of activity.)
        • LordDragonfang 1y ago
          Do we know that with certainty? Do we actually? Because my understanding is that how "thinking" works is actually still a total mystery. How is it we no for certain that the basis for the analog electric-potential-based computing done by neurons is not based on statistical prediction? Do we have actual evidence of that, or are you just doing "statistical token prediction" yourself?
          • cluckindan 1y ago
            You’re reversing the burden of proof in a similar manner as religious people often do. Absence of evidence is not evidence of absence, and so on.
            • LordDragonfang 1y ago
              I'm not reversing it lol. You're the one making a claim, the burden of evidence is on you. Absence of evidence is not evidence of absence, but it is still absence of evidence. Making a claim without any is more religious that not. After all, we know humans can't be descended from monkeys!
      • cluckindan 1y ago
        Human neurons are not reducible to arithmetic artificial neurons in a statistical model. Do not conflate them.
        • jeffhuys 1y ago
          Why not, actually?
          • cluckindan 1y ago
            Because we do not have a complete understanding of human neurons. How are we supposed to accurately model something we cannot directly observe?
            • TheDong 1y ago
              Do you also complain when someone says "Half-life 2 has great water-physics" with "Don't call it physics, we still don't understand all the physical laws of the universe, and also they use limited-precision floating-point, so it's not water-physics, it's just a bunch of math"? Like, we've agreed that "water-physics" and "cloth physics" in 3d graphics refers to a mathematical approximation of something we don't actually understand at the subatomic level (are there strings down there? Who knows). Can "thinking" in AI not refer to this intentionally false imitation that has a similar observable outward effect? Like, we're okay saying minecraft's water has "water physics", why are we not okay saying "in the AI context, thinking is a term that externally looks a bit like a human thinking, even though at a deeper layer it's unrelated"? Or is thinking special, is it like "soul" and we must defend the word with our life else we lose our humanity? If I say "that building's been thinking about falling over for 50 years", did I commit a huge faux pas against my humanity?
              • cluckindan 1y ago
                That’s comparing apples to oranges. Nobody is going to be making a real cruise ship based on game water physics simulations. In such a task, better water simulations are used. We have those, because we can directly observe the behavior of water under different conditions. It’s okay because the people doing it are explicitly aware that they are using simulation. AI will get used in real decisions affecting other people, and the people doing those decisions will be influenced by the terminology we choose to use.
    • LordDragonfang 1y ago
      "Thinking" is a term of art referring to the hidden/internal output of "reasoning" models where they output "chain of thought" before giving an answer[1]. This technique and name stem from the early observation that LLMs do better when explicitly told to "think step by step"[2]. Hope that helps clarify things for you for future constructive discussion. [1] https://arxiv.org/html/2410.10630v1 https://arxiv.org/html/2410.10630v1 [2] https://arxiv.org/pdf/2205.11916 https://arxiv.org/pdf/2205.11916
      • bobsomers 1y ago
        We are aware of the term of art. The point that was trying to be made, which I agree with, is that anthropomorphizing a statistical model isn’t actually helpful. It only serves to confuse laypersons into assuming these models are capable of a lot more than they really are. That’s perfect if you’re a salesperson trying to dump your bad AI startup onto the public with an IPO, but unhelpful for pretty much any other reason, especially true understanding of what’s going on.
        • LordDragonfang 1y ago
          If that was their point, it would have been more constructive to actually make it. To your point, it's only anthropomorphization if you make the anthrocentric assumption that "thinking" refers to something that only humans can do.[1] And I don't think it confuses laypeople, when literally telling it to "think" achieves the very similar results as in humans - it produces output that someone provided it out-of-context would easily identify as "thinking out loud", and improves the accuracy of results like how... thinking does. The best mental model of RLHF'd LLMs that I've seen is that they are statistical models "simulating"[1] how a human-like character would respond to a given natural-language input. To calculate the statistically "most likely" answer that an intelligent creature would give to a non-trivial question, with any sort of accuracy, you need emergent effects which look an awful like like a (low fidelity) simulation of intelligence. This includes simulating "thought". (And the distinction between "simulating thinking" and "thinking" is a distinction without a difference given enough accuracy) I'm curious as to what "capabilities" you think the layperson is misled about, because if anything they tend to exceed layperson understanding IME. And I'm curious what mental model you have of LLMs that provides more "true understanding" of how a statistical model can generate answers that appear nowhere in its training. [1] It also begs the question of whether there exists a clear and narrow definition of what "thinking" is that everyone can agree on. I suspect if you ask five philosophers you'll get six different answers, as the saying goes. [2] https://www.astralcodexten.com/p/janus-simulators https://www.astralcodexten.com/p/janus-simulators
    • boredhedgehog 1y ago
      These kind of comments are the equivalent of going to dog owners' forums, analyzing word choices in every post and warning the dog owners about the dangers of anthropomorphizing their pets, an effort as accurate as it is boorish and ineffectual.
      • cluckindan 1y ago
        [flagged]
      • cluckindan 1y ago
        Dogs will not be quite as widely influencing decisions concerning other people.
  • reissbaker 1y ago
    Magistral Small seems wayyy too heavy-handed with its RL to me: \boxed{Hey! How can I help you today?} They clearly rewarded the \boxed{...} formatting during their RL training, since it makes it easier to naively extract answers to math problems and thus verify them. But Magistral uses it for pretty much everything, even when it's inappropriate (in my own testing as well). It also forgets to <think> unless you use their special system prompt reminding it to. Honestly a little disappointing. It obviously benchmarks well, but it seems a little overcooked on non-benchmark usage.