7 ms·

Forcing reasoning is analogous to requiring a student to show their work when solving a problem if im understanding the paper correctly. > you’d have to either

Forcing reasoning is analogous to requiring a student to show their work when solving a problem if im understanding the paper correctly. > you’d have to either memorize the entire answer before speaking or come up with a simple pattern you could do while reciting that takes significantly less brainpower This part i dont understand. Why would coming up with an algorithm (e.g. a simple pattern) and reciting it be impossible? The paper doesnt mention the models coming up with the algorithm at all AFAIK. If the model was able to come up with the pattern required to solve the puzzles and then also execute (e.g. recite) the pattern, then that'd show understanding. However the models didn't. So if the model can answer the same question for small inputs, but not for big inputs, then doesnt that imply the model is not finding a pattern for solving the answer but is more likely pulling from memory? Like, if the model could tell you fibbonaci numbers when n=5 but not when n=10, that'd imply the numbers are memorized and the pattern for generation of numbers is not understood.

qarl 1y ago
> The paper doesnt mention the models coming up with the algorithm at all AFAIK. And that's because they specifically hamstrung their tests so that the LLMs were not "allowed" to generate algorithms. If you simply type "Give me the solution for Towers of Hanoi for 12 disks" into chatGPT it will happily give you the answer. It will write program to solve it, and then run that program to produce the answer. But according to the skeptical community - that is "cheating" because it's using tools. Nevermind that it is the most effective way to solve the problem. https://chatgpt.com/share/6845f0f2-ea14-800d-9f30-115a3b644ed4 https://chatgpt.com/share/6845f0f2-ea14-800d-9f30-115a3b644e...
- zoul 1y ago
  This is not about finding the most effective solution, it’s about showing that they “understand” the problem. Could they write the algorithm if it were not in their training set?
  - boredhedgehog 1y ago
    If that's the point, shouldn't they ask the model to explain the principle for any number of discs? What's the benefit of a concrete application?
    - johnecheck 1y ago
      Because that would prove absolutely nothing. There are numerous examples of tower of Hanoi explanations in the training set.
      - elbear 1y ago
        How do you check that a human understood it and not simply memorised different approaches?
        YeGoblynQueenne 1y ago
        You ask them to solve several instances of the problem?
        godelski 1y ago
        It's hard. But usually we ask several variations and make them show their work. But a human also isn't an LLM. It is much harder for them to just memorize a bunch of things, which makes evaluation easier. But they also get tired and hungry, which makes evaluation harder ¯\_(ツ)_/¯
        elbear 1y ago
        If we're talking about solving an equation, for example, it's not hard to memorize. Actually, that's how most students do it, they memorize the steps and what goes where[1]. But they don't really know why the algorithm works the way it does. That's what I meant by understanding. [1] In learning psychology there is something called the interleaving effect. What it says is that you solve several problems of the same kind, you start to do it automatically after the 2nd or the 3rd problem, so you stop really learning. That's why you should interleave problems that are solved with different approaches/algorithms, so you don't do things on autopilot.
- Too 1y ago
  How can one know that's not coming from the pre-trained data. The paper is trying to evaluate whether the LLM has general problem solving ability.
jsnell 1y ago
The paper doesn't mention it because either the researchers did not care to check the outputs manually, or reporting what was in the outputs would have made it obvious what their motives were. When this research has been reproduced, the "failures" on the Tower of Hanoi are the model printing out a bunch of steps, saying there is no point in doing it thousands of times more. And they they'd either output an the algorithm for printing the rest in words or code
- godelski 1y ago
  It's really easy to make a billion dollars. Just make a really useful app and sell it. There's no point explaining the rest since it's so trivial.
  - jsnell 1y ago
    That seems like a complete non sequitur. This is the model explaining the rest. Obviously the explanation is not very interesting since the Towers of Hanoi is not an interesting problem. But that's on the researches for choosing something with a trivial algorithm if their goal was to test reasoning abilities.
    - godelski 1y ago
      I'm replying to this > the model printing out a bunch of steps, saying there is no point in doing it thousands of times more.
      - jsnell 1y ago
        Ok, but the very next sentence was: > And they they'd either output an the algorithm for printing the rest in words or code. So clearly you already knew that your strawman was not relevant. Why try it anyway?
        godelski 1y ago
        Because that wasn't the task given to them. It's like giving a student a test and you asking them to solve an equation and they give you the general form. It's incomplete

qarl 1y ago

> The paper doesnt mention the models coming up with the algorithm at all AFAIK. And that's because they specifically hamstrung their tests so that the LLMs were not "allowed" to generate algorithms. If you simply type "Give me the solution for Towers of Hanoi for 12 disks" into chatGPT it will happily give you the answer. It will write program to solve it, and then run that program to produce the answer. But according to the skeptical community - that is "cheating" because it's using tools. Nevermind that it is the most effective way to solve the problem. https://chatgpt.com/share/6845f0f2-ea14-800d-9f30-115a3b644ed4 https://chatgpt.com/share/6845f0f2-ea14-800d-9f30-115a3b644e...

zoul 1y ago
This is not about finding the most effective solution, it’s about showing that they “understand” the problem. Could they write the algorithm if it were not in their training set?
- boredhedgehog 1y ago
  If that's the point, shouldn't they ask the model to explain the principle for any number of discs? What's the benefit of a concrete application?
  - johnecheck 1y ago
    Because that would prove absolutely nothing. There are numerous examples of tower of Hanoi explanations in the training set.
    - elbear 1y ago
      How do you check that a human understood it and not simply memorised different approaches?
      - YeGoblynQueenne 1y ago
        You ask them to solve several instances of the problem?
      - godelski 1y ago
        It's hard. But usually we ask several variations and make them show their work. But a human also isn't an LLM. It is much harder for them to just memorize a bunch of things, which makes evaluation easier. But they also get tired and hungry, which makes evaluation harder ¯\_(ツ)_/¯
        elbear 1y ago
        If we're talking about solving an equation, for example, it's not hard to memorize. Actually, that's how most students do it, they memorize the steps and what goes where[1]. But they don't really know why the algorithm works the way it does. That's what I meant by understanding. [1] In learning psychology there is something called the interleaving effect. What it says is that you solve several problems of the same kind, you start to do it automatically after the 2nd or the 3rd problem, so you stop really learning. That's why you should interleave problems that are solved with different approaches/algorithms, so you don't do things on autopilot.
Too 1y ago
How can one know that's not coming from the pre-trained data. The paper is trying to evaluate whether the LLM has general problem solving ability.

jsnell 1y ago

The paper doesn't mention it because either the researchers did not care to check the outputs manually, or reporting what was in the outputs would have made it obvious what their motives were. When this research has been reproduced, the "failures" on the Tower of Hanoi are the model printing out a bunch of steps, saying there is no point in doing it thousands of times more. And they they'd either output an the algorithm for printing the rest in words or code

godelski 1y ago
It's really easy to make a billion dollars. Just make a really useful app and sell it. There's no point explaining the rest since it's so trivial.
- jsnell 1y ago
  That seems like a complete non sequitur. This is the model explaining the rest. Obviously the explanation is not very interesting since the Towers of Hanoi is not an interesting problem. But that's on the researches for choosing something with a trivial algorithm if their goal was to test reasoning abilities.
  - godelski 1y ago
    I'm replying to this > the model printing out a bunch of steps, saying there is no point in doing it thousands of times more.
    - jsnell 1y ago
      Ok, but the very next sentence was: > And they they'd either output an the algorithm for printing the rest in words or code. So clearly you already knew that your strawman was not relevant. Why try it anyway?
      - godelski 1y ago
        Because that wasn't the task given to them. It's like giving a student a test and you asking them to solve an equation and they give you the general form. It's incomplete