6 ms·

Isn't that what no LLM can provide: being free of hallucinations?

svara 9mo ago
Yes, they'll probably not go away, but it's got to be possible to handle them better. Gemini (the app) has a "mitigation" feature where it tries to to Google searches to support its statements. That doesn't currently work properly in my experience. It also seems to be doing something where it adds references to statements (With a separate model? With a second pass over the output? Not sure how that works.). That works well where it adds them, but it often doesn't do it.
- deleted 9mo ago
  [deleted]
- intended 9mo ago
  Doubt it. I suspect it’s fundamentally not possible in the spirit you intend it. Reality is perfectly fine with deception and inaccuracy. For language to magically be self constraining enough to only make verified statements is… impossible.
  - svara 9mo ago
    Take a look at the new experimental AI mode in Google scholar, it's going in the right direction. It might be true that a fundamental solution to this issue is not possible without a major breakthrough, but I'm sure you can get pretty far with better tooling that surfaces relevant sources, and that would make a huge difference.
    - intended 9mo ago
      So lets run it through the rubric test - What’s your level of expertise in this domain or subject? How did you use it? What were your results? It’s basically gauging expertise vs usage to pin down the variance that seems endemic to LLM utility anecdotes/examples. For code examples I also ask which language was used, the submitters familiarity with the language, their seniority/experience and familiarity with the domain.
      - svara 9mo ago
        A lot of words to call me stupid ;) You seem to have put me in some convenient mental box of yours, I don't know which one.
        intended 9mo ago
        Oh heck no! Definitely no! I am genuinely asking, because I think one of the biggest determinants of utility obtained from LLMs is the operator. Damn, I didn’t consider that it could be read that way. I am sorry for how it came across.
kyletns 9mo ago
For the record, brains are also not free of hallucinations.
- delaminator 9mo ago
  That’s not a very useful observation though is it? The purpose of mechanisation is to standardise and over the long term reduce errors to zero. Otoh “The final truth is there is no truth”
  - michaelscott 9mo ago
    A lot of mechanisation, especially in the modern world, is not deterministic and is not always 100% right; it's a fundamental "physics at scale" issue, not something new to LLMs. I think what happened when they first appeared was that people immediately clung to a superintelligence-type AI idea of what LLMs were supposed to do, then realised that's not what they are, then kept going and swung all the way over to "these things aren't good at anything really" or "if they only fix this ONE issue I have with them, they'll actually be useful"
    - delaminator 9mo ago
      That's why I said tend to zero error. I'm a Six Sigma guy. We take accurate over precise.
- rimeice 9mo ago
  I still don’t really get this argument/excuse for why it’s acceptable that LLMs hallucinate. These tools are meant to support us, but we end up with two parties who are, as you say, prone to “hallucination” and it becomes a situation of the blind leading the blind. Ideally in these scenarios there’s at least one party with a definitive or deterministic view so the other party (i.e. us) at least has some trust in the information they’re receiving and any decisions they make off the back of it.
  - deleted 9mo ago
    [deleted]
arw0n 9mo ago
I think the better word is confabulation; fabricating plausible but false narratives based on wrong memory. Fundamentally, these models try to produce plausible text. With language models getting large, they start creating internal world models, and some research shows they actually have truth dimensions. [0] I'm not an expert on the topic, but to me it sounds plausible that a good part of the problem of confabulation comes down to misaligned incentives. These models are trained hard to be a 'helpful assistant', and this might conflict with telling the truth. Being free of hallucinations is a bit too high a bar to set anyway. Humans are extremely prone to confabulations as well, as can be seen by how unreliable eye witness reports tend to be. We usually get by through efficient tool calling (looking shit up), and some of us through expressing doubt about our own capabilities (critical thinking). [0] https://arxiv.org/abs/2407.12831 https://arxiv.org/abs/2407.12831
- svara 9mo ago
  That's right - it does seem to have to do with trying to be helpful. One demo of this that reliably works for me: Write a draft of something and ask the LLM to find the errors. Correct the errors, repeat. It will never stop finding a list of errors! The first time around and maybe the second it will be helpful, but after you've fixed the obvious things, it will start complaining about things that are perfectly fine, just to satisfy your request of finding errors.
  - thunky 9mo ago
    > It will never stop finding a list of errors! Not my experience. I find after a couple of rounds it tells me it's perfect.
- Tepix 9mo ago
  > false narratives based on wrong memory I don't think "wrong memory" is accurate, it's missing information and doesn't know it or is trained not to admit it. Checkout the Dwarkesh Podcast episode https://www.dwarkesh.com/p/sholto-trenton-2 https://www.dwarkesh.com/p/sholto-trenton-2 starting at 1:45:38 Here is the relevant quote by Trenton Bricken from the transcript: One example I didn't talk about before with how the model retrieves facts: So you say, "What sport did Michael Jordan play?" And not only can you see it hop from like Michael Jordan to basketball and answer basketball. But the model also has an awareness of when it doesn't know the answer to a fact. And so, by default, it will actually say, "I don't know the answer to this question." But if it sees something that it does know the answer to, it will inhibit the "I don't know" circuit and then reply with the circuit that it actually has the answer to. So, for example, if you ask it, "Who is Michael Batkin?" —which is just a made-up fictional person— it will by default just say, "I don't know." It's only with Michael Jordan or someone else that it will then inhibit the "I don't know" circuit. But what's really interesting here and where you can start making downstream predictions or reasoning about the model, is that the "I don't know" circuit is only on the name of the person. And so, in the paper we also ask it, "What paper did Andrej Karpathy write?" And so it recognizes the name Andrej Karpathy, because he's sufficiently famous, so that turns off the "I don't know" reply. But then when it comes time for the model to say what paper it worked on, it doesn't actually know any of his papers, and so then it needs to make something up. And so you can see different components and different circuits all interacting at the same time to lead to this final answer.
svara 9mo ago
A part of it is reproducing incorrect information in the training data as well. One area that I've found to be a great example of this is sports science. Depending on how you ask, you can get a response lifted from scientific literature, or the bro science one, even in the course of the same discussion. It makes sense, both have answers to similar questions and are very commonly repeated online.
SecretDreams 9mo ago
Find me a human that doesn't occasionally talk out of their ass =[

svara 9mo ago

Yes, they'll probably not go away, but it's got to be possible to handle them better. Gemini (the app) has a "mitigation" feature where it tries to to Google searches to support its statements. That doesn't currently work properly in my experience. It also seems to be doing something where it adds references to statements (With a separate model? With a second pass over the output? Not sure how that works.). That works well where it adds them, but it often doesn't do it.

deleted 9mo ago
[deleted]
intended 9mo ago
Doubt it. I suspect it’s fundamentally not possible in the spirit you intend it. Reality is perfectly fine with deception and inaccuracy. For language to magically be self constraining enough to only make verified statements is… impossible.
- svara 9mo ago
  Take a look at the new experimental AI mode in Google scholar, it's going in the right direction. It might be true that a fundamental solution to this issue is not possible without a major breakthrough, but I'm sure you can get pretty far with better tooling that surfaces relevant sources, and that would make a huge difference.
  - intended 9mo ago
    So lets run it through the rubric test - What’s your level of expertise in this domain or subject? How did you use it? What were your results? It’s basically gauging expertise vs usage to pin down the variance that seems endemic to LLM utility anecdotes/examples. For code examples I also ask which language was used, the submitters familiarity with the language, their seniority/experience and familiarity with the domain.
    - svara 9mo ago
      A lot of words to call me stupid ;) You seem to have put me in some convenient mental box of yours, I don't know which one.
      - intended 9mo ago
        Oh heck no! Definitely no! I am genuinely asking, because I think one of the biggest determinants of utility obtained from LLMs is the operator. Damn, I didn’t consider that it could be read that way. I am sorry for how it came across.

kyletns 9mo ago

For the record, brains are also not free of hallucinations.

delaminator 9mo ago
That’s not a very useful observation though is it? The purpose of mechanisation is to standardise and over the long term reduce errors to zero. Otoh “The final truth is there is no truth”
- michaelscott 9mo ago
  A lot of mechanisation, especially in the modern world, is not deterministic and is not always 100% right; it's a fundamental "physics at scale" issue, not something new to LLMs. I think what happened when they first appeared was that people immediately clung to a superintelligence-type AI idea of what LLMs were supposed to do, then realised that's not what they are, then kept going and swung all the way over to "these things aren't good at anything really" or "if they only fix this ONE issue I have with them, they'll actually be useful"
  - delaminator 9mo ago
    That's why I said tend to zero error. I'm a Six Sigma guy. We take accurate over precise.
rimeice 9mo ago
I still don’t really get this argument/excuse for why it’s acceptable that LLMs hallucinate. These tools are meant to support us, but we end up with two parties who are, as you say, prone to “hallucination” and it becomes a situation of the blind leading the blind. Ideally in these scenarios there’s at least one party with a definitive or deterministic view so the other party (i.e. us) at least has some trust in the information they’re receiving and any decisions they make off the back of it.
- deleted 9mo ago
  [deleted]

arw0n 9mo ago

I think the better word is confabulation; fabricating plausible but false narratives based on wrong memory. Fundamentally, these models try to produce plausible text. With language models getting large, they start creating internal world models, and some research shows they actually have truth dimensions. [0] I'm not an expert on the topic, but to me it sounds plausible that a good part of the problem of confabulation comes down to misaligned incentives. These models are trained hard to be a 'helpful assistant', and this might conflict with telling the truth. Being free of hallucinations is a bit too high a bar to set anyway. Humans are extremely prone to confabulations as well, as can be seen by how unreliable eye witness reports tend to be. We usually get by through efficient tool calling (looking shit up), and some of us through expressing doubt about our own capabilities (critical thinking). [0] https://arxiv.org/abs/2407.12831 https://arxiv.org/abs/2407.12831

svara 9mo ago
That's right - it does seem to have to do with trying to be helpful. One demo of this that reliably works for me: Write a draft of something and ask the LLM to find the errors. Correct the errors, repeat. It will never stop finding a list of errors! The first time around and maybe the second it will be helpful, but after you've fixed the obvious things, it will start complaining about things that are perfectly fine, just to satisfy your request of finding errors.
- thunky 9mo ago
  > It will never stop finding a list of errors! Not my experience. I find after a couple of rounds it tells me it's perfect.
Tepix 9mo ago
> false narratives based on wrong memory I don't think "wrong memory" is accurate, it's missing information and doesn't know it or is trained not to admit it. Checkout the Dwarkesh Podcast episode https://www.dwarkesh.com/p/sholto-trenton-2 https://www.dwarkesh.com/p/sholto-trenton-2 starting at 1:45:38 Here is the relevant quote by Trenton Bricken from the transcript: One example I didn't talk about before with how the model retrieves facts: So you say, "What sport did Michael Jordan play?" And not only can you see it hop from like Michael Jordan to basketball and answer basketball. But the model also has an awareness of when it doesn't know the answer to a fact. And so, by default, it will actually say, "I don't know the answer to this question." But if it sees something that it does know the answer to, it will inhibit the "I don't know" circuit and then reply with the circuit that it actually has the answer to. So, for example, if you ask it, "Who is Michael Batkin?" —which is just a made-up fictional person— it will by default just say, "I don't know." It's only with Michael Jordan or someone else that it will then inhibit the "I don't know" circuit. But what's really interesting here and where you can start making downstream predictions or reasoning about the model, is that the "I don't know" circuit is only on the name of the person. And so, in the paper we also ask it, "What paper did Andrej Karpathy write?" And so it recognizes the name Andrej Karpathy, because he's sufficiently famous, so that turns off the "I don't know" reply. But then when it comes time for the model to say what paper it worked on, it doesn't actually know any of his papers, and so then it needs to make something up. And so you can see different components and different circuits all interacting at the same time to lead to this final answer.

svara 9mo ago

A part of it is reproducing incorrect information in the training data as well. One area that I've found to be a great example of this is sports science. Depending on how you ask, you can get a response lifted from scientific literature, or the bro science one, even in the course of the same discussion. It makes sense, both have answers to similar questions and are very commonly repeated online.

SecretDreams 9mo ago

Find me a human that doesn't occasionally talk out of their ass =[