6 ms·
This reminded me of some weird quirk/experiment I found with LLMs that I found while messing around, maybe someone can explain it or something. Open any AI cha
by unleaded 2mo ago
This reminded me of some weird quirk/experiment I found with LLMs that I found while messing around, maybe someone can explain it or something.
Open any AI chatbot that isn't cheating by connecting to the Internet (so disable web search). Claude, DeepSeek, Kimi, whatever. Ask them this question:
"What was that weird band from michigan from the 2000s that wore coloured ties"
You will probably get a wrong answer, or if you're lucky you'll get a string of wrong answers with "wait, no - it's definitely..." before it gives up. If you aren't familiar with the band the question is referring to you might be fooled into thinking it's a tough question, but it really isn't. There is only one band that could possibly meet this criteria, you can even put the question into Google search and their Wikipedia will come up as the top result.
Then, open a new convo and ask:
"Who are Tally Hall"
The AI will easily tell you that they are a band formed in Ann Arbor, Michigan in the 2000s, known for their quirky sound and their gimmick of each member wearing a colored tie, even giving the correct color for each of them most of the time. Very odd.
- ACCount37 2mo ago"The reversal curse", it rarely shows up in practice but you found a case when it did. The "knowledge landscape" an LLM uses is "directional". It's easy to reach "a quirky music band from Michigan known for colored ties" when you stand at "Tally Hall". But if you stand at "a quirky music band from Michigan known for colored ties", it's harder to reach "Tally Hall" from there. For the "latent knowledge graph" an LLM uses, A->B doesn't cause B->A. In practice, any "common" facts will have enough "traversal" in both directions that this directional biasing isn't apparent. So it only shows up on this kind of more obscure knowledge.
- deleted 2mo ago[deleted]
- horacemorace 2mo agoIf this kind of thing holds true for humans we now may understand synesthesia, perhaps we find ourselves with a large enough study pool to map out clever tricks to influence each other.
- sigmoid10 2mo agoIt certainly holds true for humans. The brain stores relational information in a sequential pattern that is not automatically reversible. One of the best examples is the alphabet. Everyone learns it in school, so the pattern A->B->C->... is trivial to recite for most people. Now, if I gave you a random letter to start with and asked you to to recite the remaining letters until Z, you'll probably find it is still pretty easy. But if I asked you to cite the letters backwards to A, most people would suddenly struggle with this task because they never learned or used the alphabet that way in school. You need to train specifically to link this kind of information backwards in your brain.
- tobr 2mo agoIs the alphabet really the same though? I don't feel like I recall it one letter at a time, as individual facts linking A to B, then B to C, etc, but more as a sound or a phrase. Not unlike recalling a melody. It just seems very different from figuring out what band someone is describing.
- deleted 2mo ago[deleted]
- dryarzeg 2mo agoNot that I'm too sure about how this applies (or whether it applies at all) to other people as well, but for me personally it's easier to recall information about the band if I'm being told the band's name instead of being told "well, they are known for coloured ties". So, there is certainly some kind of effect described in action. Now, about alphabet: again, I think it's only me, but when I try to recall it backwards, I can't do that easily. I mean, I can recall it backwards, but I need more time to do that. It's harder. I'm not sure if it's because A links to B, B to C, then C to D and not backwards, or maybe just because in school you learn alphabet from A to Z and not from Z to a - so you're kind of trained to recall it A-->Z way - but it's certainly harder for me. At the end of the day, though, I think that everyone thinks differently. Everyone is having different internal representations for concepts (such as alphabet), so it’s not surprising that this effect may work differently for different people, or not work at all.
- spacebacon 2mo ago[dead]
- kgeist 2mo agoProbably an instance of: "The Reversal Curse: LLMs trained on "A is B" fail to learn "B is A" https://arxiv.org/abs/2309.12288 https://arxiv.org/abs/2309.12288
- hasteg 2mo agoReally interesting paper, thanks for the share. The point their making in that paper reminds me of this paper some people shared around work earlier this year, https://arxiv.org/pdf/2512.14982 https://arxiv.org/pdf/2512.14982 (Prompt Repetition Improves Non-Reasoning LLMs)... I wonder how OPs question would fare (or the questions presented in the paper you posted) given double repetition.
- fc417fc802 2mo agoA relevant question for the paper you linked is what happens if instead of repeating the input prompt you repeat a filler character the same number of times.
- hasteg 2mo agoCan you elaborate on that more? Why just a filler character?
- fc417fc802 2mo agoBecause padding the output in such a manner (which keep in mind is immediately fed back as input) has been shown to increase model performance. It's one of the many reasons to question what "thinking" traces are really doing. If you consider how the attention mechanism works then a very hand wavey intuition is that despite being entirely arbitrary additional tokens should still provide the opportunity for additional information processing.
- matankal 2mo agoThis is tested in the paper (see the "padding" ablation)
- devanshp 2mo agohttps://claude.ai/share/2b0f85a2-e7b8-4f62-91a0-eca61bdeabec https://claude.ai/share/2b0f85a2-e7b8-4f62-91a0-eca61bdeabec Fable 5 on low gets the answer with web search turned off, one-shot!
- ACCount37 2mo agoAh, that big model smell. Every time someone somewhere says "an LLM can't do this", the next generation of LLMs gains one more parameter. Until that LLM can, in fact, do this.
- optimalsolver 2mo agoSo the model was updated in the 37 minutes since OP posted his comment?
- ACCount37 2mo agoJust a funny observation. Every time someone proclaims "LLMs can't do X", a bigger, badder LLM that can in fact do X shows up shortly thereafter. Clearly, Fable 5 didn't even have the decency to wait until the next model refresh cycle to show up. It was already sitting there waiting. Either the capability gains in bigger, badder models are actually unrelated to "gotchas" being discovered, or LLMs are already acquiring Skynet levels of disrespect for cause and effect.
- unleaded 2mo agoI don't think it's solved this fundamental architectural problem by itself, it will have just squeezed the edge cases thinner. It keeps happening, people find a question it gets stupidly wrong, the vendors proclaim they've fixed it, then another one gets found.
- TeMPOraL 2mo agoOr just AI denialists like to say "LLMs can't do X" even though they can and have been doing it for the past few months or more. They only get called out once the current SOTA LLMs get so good at it, that any rando can trivially and reliably falsify the claim on the spot with whatever SOTA LLM surface they have handy. Which I suspect is what happened here, given the trail of smaller / local models that successfully answer the question, too. That said, "curse of reversability" is real, as much for LLMs as it is for people.
- famouswaffles 2mo agoRecall isn't naturally bidirectional, even for humans. If you are learning vocabulary in a new language, it's common advice to practice both target > source and source > target. Doing only one-way often makes you much better recalling that single direction than both.
- appplication 2mo agoI would need further convincing that humans do not naturally tend towards bidirectional recall. Perhaps I’m just on alert anytime I see an LLM-ism that’s met with a claim that the same or similar phenomena holds true in humans as well.
- famouswaffles 2mo ago'Naturally' might not be the best word? Maybe 'Necessarily' would be better? Regardless, it's something that happens in people. Have you not or seen someone else struggle to recall a specific fact or memory until phrased or induced in a certain way? You probably could also say LLMs 'tend towards bidirectional recall' over the course of training as things that ought to be recalled both ways are reinforced to do so. In the above example, you will also eventually learn both ways with enough exposure even without explicit practice.
- jonahrd 2mo agoWhat about the fact that "What is the weird band from the early 2000s in Michigan who wore colored ties?" could be a bar trivia question (challenging enough to recall to be fun), while "Who are Tally Hall?" could not
- pants2 2mo agoOnly because one is a short answer and the other is not
- timr 2mo agoDo you mean that you don't believe that humans learning a language have the problem the parent described? Because I do, and everyone I've ever met while learning does as well. Do you mean that you don't believe the problem exists in general, because here's another example: if you give a song title, I can easily hum the opening. If you give me the opening, I cannot reliably name the song.
- deleted 2mo ago[deleted]
- mmmattt 2mo agoI tried o3, 5.3 instant and 5.5 high and they all found it instantly with search disabled.
- fasterik 2mo agoThis doesn't seem that weird to me. Talk to any human and you'll find that their ability to recall specific names and facts is very context-dependent. Phrasing a question in one way can make it hard to answer, while providing certain words or cues can instantly "jog" the memory.
- nozzlegear 2mo agoThe LLM is a machine though, and the human is not. They don't think like us. Any similarities that you see are just coincidence, not purposeful design.
- fasterik 2mo agoI still don't see why it's surprising, though. Humans and machines are subject to the same laws of physics. While we wouldn't expect LLMs to think exactly like humans, we should expect some degree of convergent evolution. There are always tradeoffs between storage capacity, accuracy, latency, relevance, fluency, etc. Expecting LLMs to be magically perfect information retrieval machines is never going to be realistic.
- eunos 2mo agodeepseek v4 pro which doesnt have search feature could answer it
- unleaded 2mo agoI think most thinking models can do it to be fair, I think when I tried this it was all with non-thinking models/modes. Wasn't trying to make a point that LLMs can't do it or anything, just thought it was weird.
- AlexDragusin 2mo agoI asked a local instance of Qwen3.5 122b: "American indie rock/pop band that formed in Ann Arbor, Michigan, in 2001 best known for their eclectic musical style, whimsical lyrics, and distinctive visual aesthetic involving matching colored ties and hats?" Got this back "The band you are describing is Tally Hall. Formed by friends at the University of Michigan, they became well-known in the mid-2000s internet era for their songs like "Good Day" and "Rooftops," as well as their signature look where each member wore a suit with a tie and fedora in a specific color (Red, Blue, Yellow, Green, and Black/White)." Update: The gpt-oss-120b also got it correct on my prompt version. Update 1: Llama 3.1 70B gets it right as well. Update 2: Deepseek V4 Flash gets it right. Update 3: Glm 4.5 Air gets it right as well Can't seem to get any smaller models to get it. So seems it could be an issue of data points on which the model can latch onto, the more the merrier, as they say. When asked the "What was that weird band from michigan from the 2000s that wore coloured ties" was not able to get it right.
- winter_blue 2mo agoThere's a shockingly unfathomable level of data compression that these local LLMs are able to remember in an encyclopedic way, so many obscure little things.
- josh-sematic 2mo agoIf I ask my kid (early elementary) what he learned last school year, I can watch him apparently genuinely trying to think of some examples and not being very successful. If I ask him something I know he learned he’ll gladly say “oh yeah!” and tell me about it.
- michaelhoney 2mo agodude, you could ask me, a grown adult, what I did last week and get a similar lack of recall
- nozzlegear 2mo agoLLMs aren't humans, humans aren't LLMs. The fact that your kid can't recall what they learned without you reminding them is natural; the fact that the LLM can't recall something without a reprompt is a failure mode.
- Ferret7446 2mo agoAnd yet (almost?) all of the quirks people keep bringing up about LLMs are also quirks that human cognition share. It's almost as if artificial neural nets share emergent behavior with natural neural nets. Pop quiz, what's the difference between natural synthesized vanillin and artificially synthesized vanillin?
- BobbyTables2 2mo agoJust today Google’s AI told me that 3**4 was 181 because I asked it for bases such that when raised to the 4th power had a 1’s digit in the leftmost place. Yeah, LLMs aren’t designed for this kind of thing but it was really confident in its assertion… it picked the example too!
- abrbhat 2mo agoP(i|j) != P(j|i)
- fxwin 2mo agoI'm not sure which chatbots you used, but OpenAI's o3, o1 and 4.1 get it right first try (used in the Playground without web search or any other tools).
- naasking 2mo agoNot sure why this is weird when you really think about it. You're saying that a multiparameter constraint satisfaction problem is harder to solve, even when there's only a single solution (one point in the total space that satisfies all constraints), than just being given that solution/point. Well of course it is!
- JacobAsmuth 2mo agoHate to break it to you but Gemini 3.1 Flash-lite with grounding disabled (through the API) gets this first try, even with thinking disabled.
- unleaded 2mo agoYes and several other models that know about the answer don't... not sure what you're trying to break
- JacobAsmuth 2mo agoThe assumption that LLMs can't answer this question. You can pay approximately $0.000001 to correctly answer this question in approximately 50 milliseconds using Google's cheapest model.
- segmondy 2mo ago[dead]