4 ms·
BS machine produces BS; in other news water is wet and sky is blue.
by tempfile 14d ago
BS machine produces BS; in other news water is wet and sky is blue.
- Spivak 14d agoWhat's surprising about this is that you can get the bullshit machine to produce correct externally validate citations. It's not particularly hard either—it's one of the first things you build when you give an LLM access to a body of documents/search. So for a large public service to whiff like this is certainly a stain on their credibility.
- delichon 14d agoOn the other hands it's a boost to their credibility that they make their mistakes easier to evaluate than their competition does. It would be worse if they had a similar error rate without openly providing references. Kudos to Perplexity for including more empirical attack surface.
- tempfile 14d ago> you can get the bullshit machine to produce correct externally validate citations How?
- MarkusQ 14d agoBy training it on a whole bunch of examples with valid external citations, to the point where it's able to hallucinate something that's valid. Of course, it won't always work, which is the point of TFA.
- tempfile 14d agoI don't think this is an answer. I'm basically claiming there is an uncontrollable error probability, and I think you agree with that. The person I'm replying to implies it can always be reduced (maybe even to zero).
- MarkusQ 12d agoAh, yes, I see. The distinction between a working clock and one that's right twice a day.
- pessimizer 14d agoI assume by forcing them to validate their citations with a deterministic tool.
- tempfile 14d agoWhat deterministic tool will validate that a citation faithfully represents the cited claim?