5 ms·

You would think that Springer did the due diligence here, but what is the value of a brand such as Springer if they let these AI slops through their cracks? Th

You would think that Springer did the due diligence here, but what is the value of a brand such as Springer if they let these AI slops through their cracks? This is an opportunity for brands to sell verifiability, i.e., that the content they are selling has been properly vetted, which was obviously not the case here.

cess11 1y ago
Why would one think that? All of the big journal publishers have had paper millers and fraudsters and endless amounts of "tortured phrases" under their names for a long, long time.
WillAdams 1y ago
Back when I was doing academic publishing I'd use a regex to find all the hyperlinks, then a script (written by a co-worker, thanks again Dan!) to determine if they were working or no. A similar approach should work w/ a DOI.
- RossBencina 1y ago
  In the past I've had GPT4 output references with valid DOIs. Problem was the DOIs were for completely different (and unrelated) works. So you'd need to retrieve the canonical title and authors for the DOI and cross check it.
  - thoroughburro 1y ago
    And then make sure the arguments and evidence it presents are as the LLM represented them to be.
    - ofjcihen 1y ago
      At which point it’s more of a hassle to use an LLM than not.
    - rixed 1y ago
      And then check that the cited article was not itself an AI piece that managed to get published.
  - cyclecycle 1y ago
    A classic case. I work on Veracity https://groundedai.company/veracity/ https://groundedai.company/veracity/ which does citation checking for academic publishers. I see stuff like this all the time in paper submissions. Publishers are inundated
    - rbanffy 1y ago
      Don’t publishers ban authors who attempt such shenanigans?
- bumby 1y ago
  Not all journals require a DOI link for each reference. Most good ones do seem to have a system to verify the reference exists and is complete; I assume there’s some automation to that process but I’d love to hear from journal editorial staff if that’s really the case.

cess11 1y ago

Why would one think that? All of the big journal publishers have had paper millers and fraudsters and endless amounts of "tortured phrases" under their names for a long, long time.

WillAdams 1y ago

Back when I was doing academic publishing I'd use a regex to find all the hyperlinks, then a script (written by a co-worker, thanks again Dan!) to determine if they were working or no. A similar approach should work w/ a DOI.

RossBencina 1y ago
In the past I've had GPT4 output references with valid DOIs. Problem was the DOIs were for completely different (and unrelated) works. So you'd need to retrieve the canonical title and authors for the DOI and cross check it.
- thoroughburro 1y ago
  And then make sure the arguments and evidence it presents are as the LLM represented them to be.
  - ofjcihen 1y ago
    At which point it’s more of a hassle to use an LLM than not.
  - rixed 1y ago
    And then check that the cited article was not itself an AI piece that managed to get published.
- cyclecycle 1y ago
  A classic case. I work on Veracity https://groundedai.company/veracity/ https://groundedai.company/veracity/ which does citation checking for academic publishers. I see stuff like this all the time in paper submissions. Publishers are inundated
  - rbanffy 1y ago
    Don’t publishers ban authors who attempt such shenanigans?
bumby 1y ago
Not all journals require a DOI link for each reference. Most good ones do seem to have a system to verify the reference exists and is complete; I assume there’s some automation to that process but I’d love to hear from journal editorial staff if that’s really the case.