8 ms·
I find it fascinating that while in other fields you see lot of theorums/results much before practical results are found. But in this forefront of innovation -
by lowyek 2y ago
I find it fascinating that while in other fields you see lot of theorums/results much before practical results are found. But in this forefront of innovation - I have hardly seen any paper discussing hallucinations and lowerbound/upperbound on that. Or may be I didn't open hacker news on that right day when it was published. Would love to understand the hallucination phenomena more deeply and the mathematics behind it.
- amelius 2y agoI don't see many deep theorems in the field of psychology either.
- deleted 2y ago[deleted]
- dennisy 2y agoNot sure if there is a great deal of maths to understand. The output of an LLM is stochastic by nature, and will read syntactical perfect, AKA a hallucination. No real way to mathematically prove this, considering there is also no way to know if the training data also had this “hallucination” inside of it.
- ben_w 2y agoI think mathematical proof is the wrong framework, in the same way that chemistry is the wrong framework for precisely quantifying and explaining how LSD causes humans to hallucinate (you can point to which receptors it binds with, but AFAICT not much more than that). Investigate it with the tools of psychologically, as suited for use on a new non-human creature we've never encountered before.
- deleted 2y ago[deleted]
- beernet 2y agoHow are 'hallucinations' a phenomenon? I have trouble with the term 'hallucination' and believe it sets the wrong narratuve. It suggests something negative or unexpected, which it absolutely is not. Language models aim at, as their name implies, modeling language. Not facts or anything alike. This is per design and you certainly don't have to be an AI researcher to grasp that. That being said, people new to the field tend to believe that these models are fact machines. In fact, they are the complete opposite.
- deleted 2y ago[deleted]
- hbn 2y ago> the hallucination phenomena There isn't really such thing as a "hallucination" and honestly I think people should be using the word less. Whether an LLM tells you the sky is blue or the sky is purple, it's not doing anything different. It's just spitting out a sequence of characters it was trained be hopefully what a user wants. There is no definable failure state you can call a "hallucination," it's operating as correctly as any other output. But sometimes we can tell either immediately or through fact checking it spat out a string of text that claims something incorrect. If you start asking an LLM for political takes, you'll get very different answers from humans about which ones are "hallucinations"
- sqeaky 2y agoYet for their value as tools the truth value of statements made by LLMs do matter.
- mortenjorck 2y agoIt is an unfortunately anthropomorphizing term for a transformer simply operating as designed, but the thing it's become a vernacular shorthand for, "outputting a sequence of tokens representing a claim that can be uncontroversially disproven," is still a useful concept. There's definitely room for a better label, though. "Empirical mismatch" doesn't quite have the same ring as "hallucination," but it's probably a more accurate place to start from.
- NovemberWhiskey 2y ago>"outputting a sequence of tokens representing a claim that can be uncontroversially disproven," is still a useful concept. Sure, but that would require semantic mechanisms rather than statistical ones.
- riwsky 2y agoStatistics has a semantics all its own
- hbn 2y ago
- cainxinth 2y agoNot a paper, but a startup called Vectara claimed to be investigating LLM hallucination/ confabulation rates last year: https://www.nytimes.com/2023/11/06/technology/chatbots-hallucination-rates.html https://www.nytimes.com/2023/11/06/technology/chatbots-hallu...
- lowyek 2y agothank you for sharing this!
- eskibars 2y agoFYI, I work at Vectara and can answer any questions. For us, we treat hallucinations as the ability to accurately respond in an "open book" format for retrieval augmented generation (RAG) applications specifically. That is, given a set of information retrieved (X), does the LLM-produced summary: 1. Include any "real" information not contained in X? If "yes," it's a hallucination, even if that information is general knowledge. We see this as an important way to classify hallucinations in a RAG+summary context because enterprises have told us they don't want the LLMs "reading between the lines" to infer things. To pick an absurd/extreme case to show a point, the case of a genetic research firm, say, using CRISPR and finding they can create a purple zebra, if the retrieval system in the RAG bits says "zebras can be purple" due to their latest research, we don't want the LLM to override that knowledge with its knowledge that zebras are only ever black/white/brown. We'd treat that as a hallucination. 2. On the extreme opposite end, an easy way to avoid hallucinating would be for the LLM to say "I don't know" for everything thereby avoiding hallucinating by avoiding answering all questions. That has other obvious negative effects, so we also evaluate LLMs for their ability to answer. We look at the factual consistency, answer rate, summary length, and some other metrics internally to focus prompt engineering, model selection, and model training: https://github.com/vectara/hallucination-leaderboard https://github.com/vectara/hallucination-leaderboard
- cootsnuck 2y agoGreat repo, glad y'all are looking into this. So am I reading correctly that Intel has a 7B model that doesn't remarkably well with not hallucinating??
- deleted 2y ago[deleted]