6 ms·

Three years ago, people were saying "LLMs just generate plausible-sounding text, they don't understand the notion of truth so they can't do verifiable work like

Three years ago, people were saying "LLMs just generate plausible-sounding text, they don't understand the notion of truth so they can't do verifiable work like math proofs."

bigyabai 1mo ago
> they don't understand the notion of truth so they can't do verifiable work like math proofs. Nobody that understands automated proof checking was claiming that.
jdub 1mo ago
You understand that those are different kinds of "truth", right?
edot 1mo ago
Right, which was true at the time. So hundreds of billions of dollars have been poured into making LLMs better at these tasks via pretraining, RL, RLHF, post training, etc. again all with something verifiable in the loop. In order to improve the thing in the loop, the loop itself needs to be verifiable. There have only been a few thousand wars, and they’re all different and all different in the world in which they occurred. The dimensionality is absurd, which is not a problem for LLMs if there’s enough data, but in this case there isn’t.
cure_42 1mo ago
They still can't. But very smart humans constructed ways to use the monkeys with typewriters (with a statistical advantage) to find correct answers to problems where they already knew how to verify the answer.
- eru 1mo ago
  You say it like it's not an achievement.
  - strangegecko 1mo ago
    No, he says it's a different category of problem, and ability to solve one doesn't carry over to the other and it doesn't imply intelligent understanding.
- jaggederest 1mo ago
  Which problems are not amenable to that approach?
  - boomlinde 1mo ago
    Problems where verification of the proposed solution has a high cost or high risk.
  - basooza 1mo ago
    Most of them.
  - cure_42 1mo ago
    Unfortunately, most problems aren't. You need a problem where you both know what the solution looks like or can otherwise very quickly and efficiently determine that a solution is correct, but at the same time can't work out a correct solution with a similar amount of effort/time/cost as it took to determine how to verify a solution. Most problems don't match that criteria. You usually either have a problem with a known method of solving, or you have a problem with no clear way of verifying the solution besides the act of finding the solution itself which would involve in some way proving it is correct, or you have a problem where verifying a solution takes a very long time or has a high cost or even can't be done more than once, so you need to try to determine the best solution without being able to actually test or verify. Basically all problems just don't fit the "hard to solve but easy to verify" criteria to a degree that makes llms a good fit. On the other hand, there are so many problems that even a tiny fraction is a relatively large number.
jibal 1mo ago
> they don't understand the notion of truth so they can't do verifiable work like math proofs." a) No one ever said that. b) Your comment shows a lack of understanding of the notion of truth.

Three years ago, people were saying "LLMs just generate plausible-sounding text, they don't understand the notion of truth so they can't do verifiable work like math proofs."

bigyabai 1mo ago

> they don't understand the notion of truth so they can't do verifiable work like math proofs. Nobody that understands automated proof checking was claiming that.

jdub 1mo ago

You understand that those are different kinds of "truth", right?

edot 1mo ago

Right, which was true at the time. So hundreds of billions of dollars have been poured into making LLMs better at these tasks via pretraining, RL, RLHF, post training, etc. again all with something verifiable in the loop. In order to improve the thing in the loop, the loop itself needs to be verifiable. There have only been a few thousand wars, and they’re all different and all different in the world in which they occurred. The dimensionality is absurd, which is not a problem for LLMs if there’s enough data, but in this case there isn’t.

cure_42 1mo ago

They still can't. But very smart humans constructed ways to use the monkeys with typewriters (with a statistical advantage) to find correct answers to problems where they already knew how to verify the answer.

eru 1mo ago
You say it like it's not an achievement.
- strangegecko 1mo ago
  No, he says it's a different category of problem, and ability to solve one doesn't carry over to the other and it doesn't imply intelligent understanding.
jaggederest 1mo ago
Which problems are not amenable to that approach?
- boomlinde 1mo ago
  Problems where verification of the proposed solution has a high cost or high risk.
- basooza 1mo ago
  Most of them.
- cure_42 1mo ago
  Unfortunately, most problems aren't. You need a problem where you both know what the solution looks like or can otherwise very quickly and efficiently determine that a solution is correct, but at the same time can't work out a correct solution with a similar amount of effort/time/cost as it took to determine how to verify a solution. Most problems don't match that criteria. You usually either have a problem with a known method of solving, or you have a problem with no clear way of verifying the solution besides the act of finding the solution itself which would involve in some way proving it is correct, or you have a problem where verifying a solution takes a very long time or has a high cost or even can't be done more than once, so you need to try to determine the best solution without being able to actually test or verify. Basically all problems just don't fit the "hard to solve but easy to verify" criteria to a degree that makes llms a good fit. On the other hand, there are so many problems that even a tiny fraction is a relatively large number.

jibal 1mo ago

> they don't understand the notion of truth so they can't do verifiable work like math proofs." a) No one ever said that. b) Your comment shows a lack of understanding of the notion of truth.