5 ms·
Something which I haven't been able to fully parse that perhaps someone has better insight into: aren't transformers inherently only capable of inductive reason
by mcfry 11mo ago
Something which I haven't been able to fully parse that perhaps someone has better insight into: aren't transformers inherently only capable of inductive reasoning? In order to actually progress to AGI, which is being promised at least as an eventuality, don't models have to be capable of deduction? Wouldn't that mean fundamentally changing the pipeline in some way? And no, tools are not deduction. They are useful patches for the lack of deduction.
Models need to move beyond the domain of parsing existing information into existing ideas.
- hammock 11mo agoThey can induct just can’t generate new ideas. Its not going to discover a new quark without a human in the loop somewhere
- nightshift1 11mo agomaybe that's a good thing after all.
- eli_gottlieb 11mo agoThat sounds like a category mistake to me. A proof assistant or logic-programming system performs deduction, and just strapping one of those to an LLM hasn't gotten us to "AGI".
- mcfry 11mo agoA proof assistant is a verifier, and a tool so therefor a patch, so I really fail to see how that could be understood as the LLM having deduction.
- energy123 11mo agoI don't see any reason to think that transformers are not capable of deductive reasoning. Stochasticity doesn't rule out that ability. It just means the model might be wrong in its deduction, just like humans are sometimes wrong.
- mcfry 11mo agoBut it can't actually deduce, can it? If 136891438 * 1294538 isn't in the training data, it won't be able to give you a valid answer using the model itself. There's no process. It has to offload that task to a tool, which will then calculate and return. Further, any offloading needs to be manually defined at some point. You could maybe give it a way to define its own tools, but even then they would still be defined by what has come before.