7 ms·
>this error does make me pause to wonder how much of the rest of the paper used AI assistance And this is what's operative here. The error spotted, the entire
by i_am_proteus 8mo ago
>this error does make me pause to wonder how much of the rest of the paper used AI assistance
And this is what's operative here. The error spotted, the entire class of error spotted, is easily checked/verified by a non-domain expert. These are the errors we can confirm readily, with obvious and unmistakable signature of hallucination.
If these are the only errors, we are not troubled. However: we do not know if these are the only errors, they are merely a signature that the paper was submitted without being thoroughly checked for hallucinations. They are a signature that some LLM was used to generate parts of the paper and the responsible authors used this LLM without care.
Checking the rest of the paper requires domain expertise, perhaps requires an attempt at reproducing the authors' results. That the rest of the paper is now in doubt, and that this problem is so widespread, threatens the validity of the fundamental activity these papers represent: research.
- ls612 8mo agoGoogle scholar and the vagaries of copy/paste errors has mangled bibitex ever since it became a thing, a single citation with these sorts of errors may not even be AI, just “normal” mistakes.
- fn-mote 8mo agoThis seems like finding spelling errors and using them to cast the entire paper into doubt. I am unconvinced that the particular error mentioned above is a hallucination, and even less convinced that it is a sign of some kind of rampant use of AI. I hope to find better examples later in the comment section.
- j2kun 8mo agoI actually believe it was an AI hallucination, but I agree with you that it seems the problem is far more concentrated to a few select papers (e.g., one paper made up more than 10% of the detected errors).
- gold23 8mo agoWhy don't you look at the actual article? There are several more egregious examples, e.g., the authors being cited as "John Smith and Jane Doe"
- ted_dunning 8mo agoI can see that either way. It could also be a placeholder until the actual author list is inserted. This could happen if you know the title, but not the authors and insert a temporary reference entry.
- gs17 8mo agoThe first Doe and Smith example I could give that to (the title is real and the arxiv ID they give is "arXiv:2401.00001", which is definitely placeholder), but the second one doesn't match a title and has fake URL/DOI that don't actually go anywhere. There's a few that are unambiguously placeholders, but they really should have been caught in review for a conference this high up.
- mikkupikku 8mo agoHow does a "placeholder citation" even happens? Either enter the citation properly now, or do it properly later. What role does a "placeholder citation" serve, besides giving you something to forget about and fuck up? I do not believe the placeholder citation theory at all.
- recursive 8mo agoWhat's the big deal with one dead canary? This coal mine's productivity is at record highs!
- hojinkoh 8mo ago> This seems like finding spelling errors and using them to cast the entire paper into doubt. Well, to be fair, I did encounter this from actual human peer reviewers before the whole LLM thing. People do that.
- jvanderbot 8mo agoThe problem is, 10 years ago when I was still publishing even I would let an incorrect citation go through b/c of an old bibtex file or some such.
- 0xWTF 8mo agoYeah, errors of omission are so common that "Errors and Omissions" is a category of professional liability insurance.
- neilv 8mo ago> If these are the only errors, we are not troubled. However: we do not know if these are the only errors, they are merely a signature that the paper was submitted without being thoroughly checked for hallucinations. They are a signature that some LLM was used to generate parts of the paper and the responsible authors used this LLM without care. I am troubled by people using an LLM at all to write academic research papers. It's a shoddy, irresponsible way to work. And also plagiarism, when you claim authorship of it. I'd see a failure of the 'author' to catch hallucinations, to be more like a failure to hide evidence of misconduct. If academic venues are saying that using an LLM to write your papers is OK ("so long as you look it over for hallucinations"?), then those academic venues deserve every bit of operational pain and damaged reputation that will result.
- mapontosevenths 8mo ago> It's a shoddy, irresponsible way to work. And also plagiarism, when you claim authorship of it. It reminds me of kids these days and their fancy calculators! Those new fangled doohickeys just aren't reliable, and the kids never realize that they won't always have a calculator on them! Everyone should just do it the good old fashioned way with slide rules! Or these darn kids and their unreliable sources like Wikipedia! Everyone knows that you need a nice solid reliable source that's made out of dead trees and fact checked but up to 3 paid professionals!
- neilv 8mo agoAnnoying dismissal. In an academic paper, you condense a lot of thinking and work, into a writeup. Why would you blow off the writeup part, and impose AI slop upon the reviewers and the research community?
- HKH2 8mo agoI don't necessarily disagree, but researchers are not required to be good communicators. An academic can lead their field and be a terrible lecturer. A specialist can let a generalist help explain concepts for them. They should still review the final result though. There is no excuse for not doing that.
- jasonfarnon 8mo agoagree, I dont find this evidence of AI. It often happened that authors change, there are multiple venues, or I'm using an old version of the paper. We also need to see the denominator. If this google paper had this one bad citation out of 20 versus out of 60. Also everyone I know has been relying on google scholar for 10+ years. Is that AI-ish? There are definitely errors on there. If you would extrapolate from citation issues to the content in the age of LLMs, were you doing so then as well? It's the age-old debate about spelling/grammar issues in technical work. In my experience it rarely gets to the point that these errors eg from non-native speakers affect my interpretation. Others claim to infer shoddy content.
- andy12_ 8mo ago> However: we do not know if these are the only errors, they are merely a signature that the paper was submitted without being thoroughly checked for hallucinations Given how stupidly tedious and error-prone citations are, I have no trouble believing that the citation error could be the only major problem with the paper, and that it's not a sign of low quality by itself. It would be another matter entirely if we were talking about something actually important to the ideas presented in the paper, but it isn't.