5 ms·

What is "it". Gpt-5 auto? Gpt-5 pro? Deep research? These have wildly different hallucination rates.

by macrolime 11mo ago

deleted 11mo ago
[deleted]
bathtub365 11mo ago
If these rates are known it would be great for OpenAI to be open about them so customers can make an informed decision
- malfist 11mo ago
  "Known" implies that these rates are consistent and measurable. It seems to me, that this is highly unlikely to be the case
- clbrmbr 11mo ago
  OpenAI goes into great detail on hallucination rates of GPT5 models versus o3 in the GPT5 System Card [1], section 3.7. [1] https://cdn.openai.com/gpt-5-system-card.pdf#page12 https://cdn.openai.com/gpt-5-system-card.pdf#page12
- Maxatar 11mo ago
  OpenAI has published a great deal of information about hallucination rates, as have the other major LLM providers. You can't just give one single global hallucination rate since the rates depend on the different use cases and despite the abundant amount of information available to people on how to pick the appropriate tool for a given task, it seems very few people care to take the time to actually first recognize that these LLMs are tools, and that you do need to learn how to use these tools in order to be productive with them.
crazygringo 11mo ago
I use all of the current versions of ChatGPT, Gemini, and Claude. The hallucination rates are about the same as far as I can tell. It depends mostly on how niche the area is, not which model. They do seem to train on somewhat different sets of academic sources, so it's good to use them all. I'm not talking about deep research or advanced thinking modes -- those are great for some tasks but don't really add anything when you're just looking for all the sources on a subject, as opposed to a research report.
- astrange 11mo ago
  ChatGPT thinking mode is definitely the best search engine (wrapper) I've ever used, and you should be using it to find sources.

What is "it". Gpt-5 auto? Gpt-5 pro? Deep research? These have wildly different hallucination rates.

deleted 11mo ago

[deleted]

bathtub365 11mo ago

If these rates are known it would be great for OpenAI to be open about them so customers can make an informed decision

malfist 11mo ago
"Known" implies that these rates are consistent and measurable. It seems to me, that this is highly unlikely to be the case
clbrmbr 11mo ago
OpenAI goes into great detail on hallucination rates of GPT5 models versus o3 in the GPT5 System Card [1], section 3.7. [1] https://cdn.openai.com/gpt-5-system-card.pdf#page12 https://cdn.openai.com/gpt-5-system-card.pdf#page12
Maxatar 11mo ago
OpenAI has published a great deal of information about hallucination rates, as have the other major LLM providers. You can't just give one single global hallucination rate since the rates depend on the different use cases and despite the abundant amount of information available to people on how to pick the appropriate tool for a given task, it seems very few people care to take the time to actually first recognize that these LLMs are tools, and that you do need to learn how to use these tools in order to be productive with them.

crazygringo 11mo ago

I use all of the current versions of ChatGPT, Gemini, and Claude. The hallucination rates are about the same as far as I can tell. It depends mostly on how niche the area is, not which model. They do seem to train on somewhat different sets of academic sources, so it's good to use them all. I'm not talking about deep research or advanced thinking modes -- those are great for some tasks but don't really add anything when you're just looking for all the sources on a subject, as opposed to a research report.

astrange 11mo ago
ChatGPT thinking mode is definitely the best search engine (wrapper) I've ever used, and you should be using it to find sources.