5 ms·
If an LLM was trained to always say "I don't know", that'd be a useless LLM, so they're trained to NOT say that, even when they don't actually know. The LLM wa
by BariumBlue 3y ago
If an LLM was trained to always say "I don't know", that'd be a useless LLM, so they're trained to NOT say that, even when they don't actually know.
The LLM was happy to give you 'happy' but incorrect answers because it thought it'd make you the most satisfied. Kinda like a psychopath car salesman who just wants to make a sale.
- thesuperbigfrog 3y ago>> If an LLM was trained to always say "I don't know", that'd be a useless LLM, so they're trained to NOT say that, even when they don't actually know. Do LLMs "know" that they don't know? Real understanding includes acknowledging what you don't know. Can the predictive text generation of LLMs recognize that its training set did not include data?
- famouswaffles 3y agoThere's quite a lot of indication that the computation can distinguishing hallucinations. It just has no incentive to communicate this. GPT-4 logits calibration pre RLHF - https://imgur.com/a/3gYel9r https://imgur.com/a/3gYel9r Just Ask for Calibration: Strategies for Eliciting Calibrated Confidence Scores from Language Models Fine-Tuned with Human Feedback - https://arxiv.org/abs/2305.14975 https://arxiv.org/abs/2305.14975 Teaching Models to Express Their Uncertainty in Words - https://arxiv.org/abs/2205.14334 https://arxiv.org/abs/2205.14334 Language Models (Mostly) Know What They Know - https://arxiv.org/abs/2207.05221 https://arxiv.org/abs/2207.05221
- astrange 3y agoWorks for me. https://chat.openai.com/share/683a5a05-50d9-461e-872a-d16b71f1f8e7 https://chat.openai.com/share/683a5a05-50d9-461e-872a-d16b71...