11 ms·
Reasoning models reason well, until they don't
- anal_reactor 11mo agoI'm yet to see a task that AI fails at that bottom 10% of population wouldn't also fail at.
- deleted 11mo ago[deleted]
- TheOtherHobbes 11mo agoHow about keeping a conversation going with family over Thanksgiving? (Or local equivalent.)
- randomNumber7 11mo agoThis is something where the top 10% sometimes horribly fail.
- Earw0rm 11mo agoIf by task you mean the written, intellectual variety, maybe.
- deleted 11mo ago[deleted]
- layer8 11mo agoIf I have the choice of performing an intellectual task myself, or have it performed by someone from the bottom 10% of the population, I’d probably rather perform it myself.
- Der_Einzige 11mo agoWhat happens when both choices lead to you doing it yourself?
- acdha 11mo agoThe problem is consistency: AI tools usually produce output which _sounds_ like the top 10% but you have to read it carefully to find the bottom 10% parts. We’re not used to that because human performance isn’t that inconsistent and we use history and social factors: someone’s performance goes down when they’re really drunk, but they rarely show up to work in that state and it’s obvious enough that other people recognize that they shouldn’t be trusted.
- anal_reactor 11mo ago> We’re not used to that because human performance isn’t that inconsistent It is. It's very common for socially apt people to bullshit through things they don't know, or outright want to hide.
- acdha 11mo agoThat’s not inconsistent: your bluffer knows they’re making something up and is using their model of you to construct something they think you’ll believe. Someone who can do that isn’t going to suddenly forget how to count the number of letters in a word.
- anal_reactor 11mo agoYou're wrong. Counting the number of letters in a word is a significantly more difficult task than lying, both for humans and LLMs. Imagine going to a ghetto and asking people "have you ever lied to someone and had them believe the lie", and ask them to spell "continuously". Children learn to lie before they learn to spell.
- acdha 11mo ago> Counting the number of letters in a word is a significantly more difficult task than lying No, it’s not - you don’t even need to be literate to count symbols - but also consider the complexity of the second task and how many skills each requires: unlike counting letters, lying isn’t simple confabulation and requires a theory of mind and some kind of goal. A child who lies to avoid trouble is doing that because they have enough of a world model to know they are going to get in trouble for something even if they haven’t worked out yet that this is unlikely to work.
- krackers 11mo agoARC-AGI v3 is a pretty good benchmark, and it's notably different from the other ARC-AGI in that it has a "truer" human baseline (you can go play it right now and add your datapoint), and captures the act of in-context learning better as you start an unfamiliar game then master it over time. Also bottom 10% feels like a bad comparison, median human would be better. And unlike "specialized" things like programming, game playing is something almost all of us have done.
- iLoveOncall 11mo ago> [...] recent studies show that transformers and LLMs fail catastrophically once reasoning problems exceed modest complexity. We revisit these findings through the lens of large reasoning models (LRMs) -- LLMs fine-tuned with incentives for step-by-step argumentation and self-verification This was the obvious outcome of the study (don't get me wrong, obvious outcomes are still worth having research on). "LRMs" *are* just LLMs. There's no such thing as a reasoning model, it's just having an LLM write a better prompt than the human would and then sending it to the LLM again. Despite what Amodei and Altman want Wall Street to believe, they did not suddenly unlock reasoning capabilities in LLMs by essentially just running two different prompts in sequence to answer the user's question. The truly amazing thing is that reasoning models show ANY improvement at all compared to non-reasoning models, when they're the same exact thing.
- sirwhinesalot 11mo ago> The truly amazing thing is that reasoning models show ANY improvement at all compared to non-reasoning models, when they're the same exact thing. It's because they do more compute. The more tokens "spent" the better the accuracy. Same reason they spit out a paragraph of text instead of just giving a straight answer in non-reasoning mode.
- sothatsit 11mo agoWhat do you mean by reasoning? If you mean solving logic problems, then reasoning LLMs seem to pass that bar as they do very well programming and maths competitions. Reasoning LLMs can also complete problems like multiplying large numbers, which requires applying some sort of algorithm where the results cannot just be memorised. They also do this much better than standard pre-trained LLMs with no RL. So, that makes me come back to this question of what definition of reasoning do people use that reasoning models do not meet? They're not perfect, obviously, but that is not a requirement of reasoning if you agree that humans can reason. We make mistakes as well, and we also suffer under higher complexity. Perhaps they are less reliable in knowing when they have made mistakes or not than trained humans, but I wouldn't personally include reliability in my definition for reasoning (just look at how often humans make mistakes in tests). I am yet to see any serious, reasoned, arguments that suggest why the amazing achievements of reasoning LLMs in maths and programming competitions, on novel problems, does not count as "real reasoning". It seems much more that people just don't like the idea of LLMs reasoning, and so reject the idea without giving an actual reason themselves, which seems somewhat ironic to me.
- equinox_nl 11mo agoBut I also fail catastrophically once a reasoning problem exceeds modest complexity.
- monkeydust 11mo agoBut you recognise you are likely to fail and thus dont respond or redirect the problem to someone who has a greater likelihood of not failing.
- antonvs 11mo agoI’ve had models “redirect the problem to someone who has a greater likelihood of not failing”. Gemini in particular will do this when it runs into trouble. I don’t find all these claims that models are somehow worse than humans in such areas convincing. Yes, they’re worse in some respects. But when you’re talking about things related to failures and accuracy, they’re mostly superhuman. For example, how many humans can write hundred of lines of code (in seconds mind you) and regularly not have any syntax errors or bugs?
- ffsm8 11mo ago> For example, how many humans can write hundred of lines of code (in seconds mind you) and regularly not have any syntax errors or bugs? Ez, just use codegen. Also the second part (not having bugs) is unlikely to be true for the LLM generated code, whereas traditional codegen will actually generate code with pretty much no bugs.
- WesolyKubeczek 11mo agoIt’s because they generate a seeming of reasoning, and don’t actually reason! (Slams the door angrily) (stomps out angrily) (touches the grass angrily)
- samuell 11mo agoYea, a bit like a cheating student rote memorizing and copying another students technique for solving a type of problem, and failing hard as soon as there's too much variation from the original problem.
- fsloth 11mo agoYes! That said the input space of supported problems is quite large and you can configure the problem parametrs quite flexibly. I guess the issue is that what the model _actually_ provides you is this idiot savant who has pre-memorized everything without offering a clear index that would disambiguate well-supported problems from ”too difficult” (i.e. novel) ones
- brap 11mo agoWhat is to reason, if not to generate a seeming of reasoning? (tips fedora)
- hshdhdhehd 11mo agoYou said the quiet part out loud of political debate. (does something)
- brap 11mo agoI wonder if we can get models to reason in a structured and verifiable way, like we have formal logic in math.
- Frieren 11mo agoFor that, you already have classical programming. It is great at formal logic math.
- brap 11mo agoI think trying to accurately express natural language statements as values and logical steps as operators is going to be very difficult. You also need to take into account ambiguity and subtext and things like that. I actually believe it is technically possible, but is going to be very hard.
- nl 11mo agoThis is where you get the natural language tool to write the formal logic. ChatGPT knows WebPPL really well for example.
- brap 11mo agoYou will need a formal language first. Take this statement for example: >ChatGPT knows WebPPL really well What formal language can express this statement? What will the text be parsed into? Which transformations can you use to produce other truthful (and interesting) statements from it? Is this flexible enough to capture everything that can be expressed in English? The closest that comes to mind is Prolog, but it doesn’t really come close.
- nl 11mo ago> You will need a formal language first. No, that's the entire point! The LLM is the bridge between natural language and a formal specification. (WebPPL is a formal language btw. It's not unlike Prolog but is designed from the start to express lemmas probabilistically)
- alyxya 11mo agoThe key point the paper seems to make is that existing benchmarks have relatively low complexity on reasoning complexity, so they made a new dataset DeepRD with arbitrarily large reasoning complexity and demonstrated that existing models fail at a complex enough problem. Complexity is defined from the complexity of a graph created by modeling the problem as a graph and determining the traversals needed to go from some source node to a target node. My main critique is that I don't think there's evidence that this issue would persist after continuing to scale models to be larger and doing more RL. With a harness like what coding agents do these days and with sufficient tool use, I bet models could go much further on that reasoning benchmark. Otherwise, if the reasoning problem were entirely done within a single context window, it's expected that a complex enough reasoning problem would be too difficult for the model to solve.
- jeremyjh 11mo agoThe burden of evidence here is on you. They don’t need to prove LRMs can’t scale to meet these problems; their only claim is current models can’t handle these problems. Others will take this up as a challenge - and chances may be good they will overcome it. This is how science works.
- alyxya 11mo agoThey can’t claim current models aren’t able to handle these problems if they didn’t use a setup similar to coding agents like Claude Code and OpenAI Codex. Using a suboptimal setup is akin to verbally telling a person the whole reasoning problem without letting them write down notes and expecting them to memorize and solve it after only hearing it once.
- js8 11mo agoI think the explanation is pretty simple, as I said in my earlier comment: https://news.ycombinator.com/item?id=44904107 https://news.ycombinator.com/item?id=44904107 I also believe the problem is we don't know what we want: https://news.ycombinator.com/item?id=45509015 https://news.ycombinator.com/item?id=45509015 If we could make LLMs to apply a modest set of logic rules consistently, it would be a win.
- Sharlin 11mo agoThat's a pretty big "if". LLMs are by design entirely unlike GoFAI reasoning engines. It's also very debatable whether it makes any sense to try and hack LLMs into reasoning engines when you could just... use a reasoning engine. Or have the LLM to defer to one, which would play to their strength as translators.
- flimflamm 11mo agoWhat confused me is the fact that in the paper all logical steps are give. It basically check that when all relevant facts are provided explicitly as links , how far and how complex a chain can the model correctly follow before it breaks down? So it's simpler than "reasoning". This is not necessarily a bad thing as it boils down the reasoning to a simpler, more controlled sub problem.
- devlogstream 11mo agoLLMs are like students, they can reason a bit, but real understanding still takes time and practice.
- hansmayer 11mo agoWhat? The LLMs are nothing like students (or any other human for that matter).
- My_Name 11mo agoI find that they know what they know fairly well, but if you move beyond that, into what can be reasoned from what they know, they have a profound lack of ability to do that. They are good at repeating their training data, not thinking about it. The problem, I find, is that they then don't stop, or say they don't know (unless explicitly prompted to do so) they just make stuff up and express it with just as much confidence.
- ftalbot 11mo agoEvery token in a response has an element of randomness to it. This means they’re non-deterministic. Even if you set up something within their training data there is some chance that you could get a nonsense, opposite, and/or dangerous result. The chance of that may be low because of things being set up for it to review its result, but there is no way to make a non-deterministic answer fully bound to solving or reasoning anything assuredly, given enough iterations. It is designed to be imperfect.
- yuvalr1 11mo agoYou are making a wrong leap from non-deterministic process to uncontrollable result. Most of the parallel algorithms are non-deterministic. There might be no guarantee about the order of calculation or even sometimes the final absolute result. However, even when producing different final results, the algorithm can still guarantee characteristics about the result. The hard problem then is not to eliminate non-deterministic behavior, but find a way to control it so that it produces what you want.
- flavaflav2 11mo agoLife and a lot in our universe is non-deterministic. Some people assume science and mathematics are some universal truths rather than imperfect agreed upon understandings. Similarly many assume humans can be controlled through laws, penalties, prisons, propaganda, coercion, etc. But terrible things happen. Yes, if you set up the gutter-rails in your bowling lane, you can control the bowling ball unless it is thrown over those rails or in a completely different direction, but those rails are wide with LLMs by default, and the system instructions provided it aren’t rules, they are an inherently faulty way to coerce a non-deterministic system. But, yes, if there’s absolutely no way to do something, and you’re aware of every possible way a response or tool could affect things, and you have taken every possible precaution, you can make it behave. That’s not how people are using it though, and we cannot control our tendency to trust that which seems trustworthy even if we are told these things.
- nakamoto_damacy 11mo agoLLMs falter because likelihood-driven pattern completion doesn’t enforce coherence across uncertainty (probability), representation (geometry), composition (category), and search (reasoning). To get robust reasoning, we need these layers to be explicit, typed, and mutually constraining—with verification and calibrated belief updates in the loop. I was interviewed about this recently, and mentioned the great work of a professor of CS and Law who has been building the foundations for this approach. My own article about it was recently un-linked due to a Notion mishap (but available if anyone is interested - I have to publish it again) https://www.forbes.com/sites/hessiejones/2025/09/30/llms-are-short-circuiting-is-it-time-to-redefine-intelligence/ https://www.forbes.com/sites/hessiejones/2025/09/30/llms-are...
- CuriouslyC 11mo agoRichard Sutton's interview on Dwarkesh's podcast hit at this same point. The implicit world models in LLMs are insufficient.
- jampekka 11mo agoSutton still hasn't learned his own Bitter Lesson? ;)
- creativeSlumber 11mo agowhat do you mean?
- nakamoto_damacy 11mo agoNot sure why he capitalized bitter...
- jampekka 11mo agoIt was a joke referring to his essay. https://en.wikipedia.org/wiki/Bitter_lesson https://en.wikipedia.org/wiki/Bitter_lesson
- hirako2000 11mo agoHas any one ever found an ML/AI paper that make claims that RLMs can reason? When I prompt an RLM, I can see it spits out reasoning steps. But I don't find that evidence RLMs are capable of reasoning.
- Sharlin 11mo agoSemantics schemantics.
- hirako2000 11mo agoIt's a statistical imitation of a reasoning pattern, underlying mechanism is pattern matching. The ability to create a model that can determine two radically different words have strong similarity in meaning doesn't imply emergence of some generalizable, logical model that suddenly can Reason to solve novel problems. Pattern matching is a component of reason. Not === reason.
- _heimdall 11mo agoThat would require the ability to understand what happens inside the system during inference when the output is created and they can't do that today. There's no evidence to be had when we only know the inputs and outputs of a black box.
- tempfile 11mo agoI don't understand what point you are making. Doesn't the name "Reasoning language models" claim that they can reason? Why do you want to see it explicitly written down in a paper?
- hirako2000 11mo agoThis very paper sits on the assumption reasoning (to solve puzzles) is at play. It calls those LLMs RLMs. Imo the paper itself should have touched on the lack of paper discussing what's in the blackbox that makes them Reasoning LMs. It does mention some tree algorithm supposedly key to reasoning capabilities. By no means attacking the paper as its intent is to demonstrate the lack of success to even solve simple to formulate, complex puzzles. I was not making a point, I was genuinely asking in case someone knows of papers I could read on that make claims with evidence that's those RLM actually reason, and how.
- egberts1 11mo agoIt's simple. Don't ingest more than 40KB at a time into its LLM's RAG pipe and its hallucination goes way, way down. Preferably like not at the start and best not to do more than 40KB at a time at all. That's how I learned how to deal with nftables' 120KB parser_bison.y file by breaking them up into clean sections. All of a sudden, a fully-deterministic LL(1) full semantic pathway of nftables' CLI syntax appears before my very eye (and spent hours validating it): 100% and test generators now can permutate crazy test cases with relative ease. Cue in Joe Walsh's "Life's Been Good To Me".
- bob_theslob646 11mo agoWhy 40kb?
- lingrush4 11mo agoIs that really the best title the authors could come up with? Up next: "Lawn mowers are good at cutting grass until they aren't"
- andy99 11mo agoI think that would be a good title if we’d previously thought lawn mowers had solved generalized grass cutting and assumed that because one worked on my lawn that they could cut hayfields or harvest bamboo (a grass I believe) effectively.
- tekno45 11mo agoWhen the news cycle has been "lawnmowers can now do anything, throw away your kitchenaide" its a pretty relevant title.
- moritzwarhier 11mo agoFrom the abstract: > some even claiming they are capable of generalized reasoning and innovation in reasoning-intensive fields such as mathematics, physics, medicine, and law. However, by more carefully scaling the complexity of reasoning problems, we show existing benchmarks actually have limited complexity Can someone ELI5 what the definitions of reasoning and complexity are here? I see they seem to focus on graph problems and representing problems as graph problems. But I didn't completely read the paper or understand it in depth. I skimmed some parts that seem to address this question (e.g. section 5 and the Introduction), but maybe there are simpler definitions that elude me. Surely they don't mean "computational complexity"? And what exactly is "reasoning"? I'm aware of philosophical logic and strict logic that can be applied to natural language arguments. But have we already agreed on a universal scale that grades answers to questions about the physical world? Or is this about mathematical reasoning? Mixing all of this together always irks me when it comes to these AI "benchmarks". But apparently people see value in these? I know my question isn't new. To me it seems, that when we leave the mathematical realms, it quickly becomes fuzzy what correct "reasoning" should be. People can be convincing and avoid obious logical fallacies, and still make wrong conclusions... or conclusions that run counter to assumed goals.
- dcre 11mo agoEven in the mathematical/formal realm, the meaning of reasoning is not as clear as it seems. The result of the activity of reasoning may be a formal argument that can be evaluated according to well-defined rules, but the actual process your mind went through to get there is just as opaque (or more) as whatever is going on inside LLMs. It seems likely, as you suggest, that we are going to have to define reasoning in terms of ability to solve certain classes of problems but leaving the character of the process unspecified.
- kordlessagain 11mo agoWhat specific reasoning capabilities matter for what real-world applications? Nobody knows. Moreover, nobody talks about that because it's boring and non-polarizing. Instead, supposedly smart people post stupid comments that prevent anyone from understanding this paper is worthless. The paper is worthless because it has a click-bait title. Blog posts get voted down for that, why not this? The implicit claim is worthless. Failure to navigate a synthetic graph == failure to solve real world problems. False. Absolutely no connection to real world examples. Just losing the model in endless graphs.
- wavemode 11mo ago> The implicit claim is worthless. Failure to navigate a synthetic graph == failure to solve real world problems. False. This statement is the dictionary definition of attacking a strawman. Every new model that is sold to us, is sold on the basis that it performs better than the old model on synthetic benchmarks. This paper presents a different benchmark that those same LLMs perform much worse on. You can certainly criticize the methodology if the authors have erred in some way, but I'm not sure why it's hard to understand the relevance of the topic itself. If benchmarks are so worthless then go tell that to the LLM companies.
- riskable 11mo agoMy hypothesis: This is why AI is fantastic as a coding assistant but not so great at other things. A software developer—after watching an AI model fail over and over again, trying to say, fix a difficult bug—will stop and approach the issue from a different angle. They'll take a closer look at what's going on, fiddle things around by hand, and that's usually enough to get over that hump of complexity (that the AI model couldn't work its way through). We (developers) do this because it's what we've always done with our own code. Everyone's encountered a bug that they just couldn't figure out. So they search the Internet, try different implementations of the same thing, etc but nothing works. Usually, we finally solve such problems when we take a step back and look at it with a different lens. For example, just the other day—after spending far too long trying to get something working—I realized, "Fuck it! The users don't really need this feature." :thumbsup:
- acuozzo 11mo ago> AI is fantastic as a coding assistant The extent to which this is true is a rough measure of how derivative your work is, no?
- dankai 11mo agoThis is not the only paper that scales reasoning complexity / difficulty. The CogniLoad benchmark does this as well (in addition to scaling reasoning length and distractor ratio). Requiring the LLM to purely reason based on what is in the context (i.e. not based on the information its pretrained on), it finds that reasoning performance decreases significantly as problems get harder (i.e. require the LLM to hold more information in its hidden state simultaneously), but the bigger challenge for them is length. https://arxiv.org/abs/2509.18458 https://arxiv.org/abs/2509.18458 Disclaimer: I'm the primary author of CogniLoad so feel free to ask me any questions.
- kerabatsos 11mo agoHow is that different than human reasoning?
- ares623 11mo agoI’d like $500B to just be the way I am thanks.
- j45 11mo agoCompared to software that can explicitly reason, reasoning models don’t seem to reason at all. They simulate reasoning through matching patterns.