6 ms·
How do you validate this kind of work to weed out any confabulating by the LLMs?
by jessetemp 3mo ago
How do you validate this kind of work to weed out any confabulating by the LLMs?
- dominotw 3mo agoi love how gp posted a glowing review and then dipped out.
- arbitrary_name 3mo agosorry, did they owe you and any other poster something?
- lebovic 3mo agoAfter spending years on a problem, it's exciting to see it start to get more attention and move towards being meaningfully solved. But I try to limit my time on HN, and I thought someone who works on Claude Science might respond to this thread later.
- lebovic 3mo agoI assume they do hallucinate, just like with coding or finding vulnerabilities. You can try to minimize it (e.g. with a reviewer agent, which Claude Science and Biomni have), but nothing is perfect, so I limit autonomous work to verifiable problems and review it.
- ep103 3mo agoHonestly, this is how all AI should be used, in most non-trivial scenearios
- ml_more 3mo agoWhen you set up your Claude Science instance you can see that they're connecting to crossref, semantic scholar, pubmed, ArXiv, FDA. They instruct the LLM to validate citations. My testing with this technique indicates that method they seem to be using (rag with an instruction to check sources) will reduce the confabulation rate for citations from the base rate 50-60% for regular models (e.g. regular Claude) to 5-15% (depending on how they implemented it). On the one hand this is way better. On the other hand it's just good enough that your spot check will look good and your work will still contain hallucinations (which is probably worse than obviously bad). Getting to zero confabulation would require a different process. (stand-alone validation engine running in parallel in real-time which is hard but not impossible.)