5 ms·

Solving the paper submission is easy. Just hold frontal interview where the submitter defends their paper. They can't create papers every day and still be knowl

Solving the paper submission is easy. Just hold frontal interview where the submitter defends their paper. They can't create papers every day and still be knowledgeable about them in depth. We are hurling to a reality where the only noteworthy metric is human to human validation.

SpicyLemonZest 3mo ago
If a journal finds that it's getting more papers than peer reviewers are willing to go through, how does a more heavyweight, synchronous review process solve the problem? Many researchers already find peer review requests annoying, they're not going to agree to hold a bunch of video calls.
- watwut 3mo ago
  Big part of the annoyance is that journals demand basically free labor - while costing massive amount of money if you want to read them. A review call might just end up being less work then reading a lot of slop papers.
- magic_hamster 2mo ago
  If the problem is that there are too many AI slop papers, then requiring real human input and feedback can help reduce these numbers. Let's say you have to defend your paper in person before you submit it, it will cut slop submissions dramatically. The current system is just not built for a world with LLMs producing mountains of content with automation.
glial 3mo ago
Paper reviews are traditionally blinded, so the reviewer doesn't know the authorship of the paper they're reading.
- ViscountPenguin 3mo ago
  You could do a two stage process, a blinded text review and an unblinded review that can only be failed if you have a vigorous lack of knowledge about your work. I highly doubt anything like that will be implemented though.
matusp 3mo ago
With the volume of outputs in today's academia, this is simply not possible. There are conferences with tens of thousands of submitted papers, grants have hundreds of pages, etc.
- jltsiren 3mo ago
  Grants typically have at most 10 pages of actual content. The rest is mostly compliance with regulations.

SpicyLemonZest 3mo ago

If a journal finds that it's getting more papers than peer reviewers are willing to go through, how does a more heavyweight, synchronous review process solve the problem? Many researchers already find peer review requests annoying, they're not going to agree to hold a bunch of video calls.

watwut 3mo ago
Big part of the annoyance is that journals demand basically free labor - while costing massive amount of money if you want to read them. A review call might just end up being less work then reading a lot of slop papers.
magic_hamster 2mo ago
If the problem is that there are too many AI slop papers, then requiring real human input and feedback can help reduce these numbers. Let's say you have to defend your paper in person before you submit it, it will cut slop submissions dramatically. The current system is just not built for a world with LLMs producing mountains of content with automation.

glial 3mo ago

Paper reviews are traditionally blinded, so the reviewer doesn't know the authorship of the paper they're reading.

ViscountPenguin 3mo ago
You could do a two stage process, a blinded text review and an unblinded review that can only be failed if you have a vigorous lack of knowledge about your work. I highly doubt anything like that will be implemented though.

matusp 3mo ago

With the volume of outputs in today's academia, this is simply not possible. There are conferences with tens of thousands of submitted papers, grants have hundreds of pages, etc.

jltsiren 3mo ago
Grants typically have at most 10 pages of actual content. The rest is mostly compliance with regulations.