9 ms·
Updated practice for review articles and position papers in ArXiv CS category
- thomascountz 11mo agoThe HN submission title is incorrect. > Before being considered for submission to arXiv’s CS category, review articles and position papers must now be accepted at a journal or a conference and complete successful peer review. Edit: original title was "arXiv No Longer Accepts Computer Science Position or Review Papers Due to LLMs"
- stefan_ 11mo agoIsn't arXiv where you upload things before they have gone through the entire process? Isn't that the entire value, aside from some publisher cartel busting?
- jvanderbot 11mo agoAlmost all CS papers can still be uploaded, and all non-CS papers. This is a very conservative step by them.
- catlifeonmars 11mo agoAgree. Additionally, original title, "arXiv No Longer Accepts Computer Science Position or Review Papers Due to LLMs" is ambiguous. “Due to LLMs” is being interpreted as articles written by LLMs, which is not accurate.
- zerocrates 11mo agoNo, the post is definitely complaining about articles written by LLMs: "In the past few years, arXiv has been flooded with papers. Generative AI / large language models have added to this flood by making papers – especially papers not introducing new research results – fast and easy to write." "Fast forward to present day – submissions to arXiv in general have risen dramatically, and we now receive hundreds of review articles every month. The advent of large language models have made this type of content relatively easy to churn out on demand, and the majority of the review articles we receive are little more than annotated bibliographies, with no substantial discussion of open research issues." Surely a lot of them are also about LLMs: LLMs are the hot computing topic and where all the money and attention is, and they're also used heavily in the field. So that could at least partially account for why this policy is for CS papers only, but the announcement's rationale is about LLMs as producing the papers, not as their subject.
- deleted 11mo ago[deleted]
- dimava 11mo agorefined title: ArXiv CS requires peer review for surveys amid flood of AI-written ones - nothing happened to preprints - "summarization" articles always required it, they are just pointing at it out loud
- ivape 11mo agoI don’t know about this. From a pure entertainment standpoint, we may be denying ourselves a world of hilarity. LLMs + “You know Peter, I’m something of a research myself” delusions. I’d pay for this so long as people are very serious about the delusion.
- aoki 11mo agoThat’s viXra
- deleted 11mo ago[deleted]
- dang 11mo agoWe've reverted it now.
- ThrowawayTestr 11mo agoThis is hilarious. Isn't arXiv the place where everyone uploads their paper?
- anthk 11mo agoI've seen odd stuff elsewhere, too: https://pubmed.ncbi.nlm.nih.gov/18955255/ https://pubmed.ncbi.nlm.nih.gov/18955255/ https://pubmed.ncbi.nlm.nih.gov/16136218/ https://pubmed.ncbi.nlm.nih.gov/16136218/
- Maken 11mo agoarXiv was built over a good faith assumption, where a long paper meant at least the author had put some effort behind, and a every idea deserved attention. AI generated text breaks that assumption, and anybody uploading it is not acting in good faith. And it's a unequal arms race, in which generating endless slop is way cheaper than storing it, because slop generators are subsidised (by operating at a loss) but arXiv has to pay the full price for their hosting.
- j45 11mo agoHave the papers gotten that good or bad?
- Sharlin 11mo agoYep, so good that they have to be specifically reviewed because otherwise people wouldn’t believe how good they are.
- Maken 11mo agoActual papers are as good as ever. This is just trying to stop the flood of autogenated slop, if anything because arXiv hosting space is not free.
- physarum_salad 11mo agoIt is actually great because it shows how well it works as a system. Screening is really important to keep preprint quality high enough to then implement cool ideas like random peer review/automated reviews etc
- JumpCrisscross 11mo ago> we are developing a whole new method to do peer review What’s the new method?
- physarum_salad 11mo agoI mean generally working towards changing how peer review works. For example: https://prereview.org/en-us https://prereview.org/en-us Anecdotally, a lot of researchers will run their paper pdfs through an AI iteration or two during drafting which also (kinda but not really) counts as a self-review. Although that is not comparable to peer review ofc.
- candiddevmike 11mo agoI've seen quite a few preprints posted on HN with clearly fantastical claims that only seem to reinforce or ride the coattails of the current hype cycle. It's no longer research, it's becoming "top of funnel thought leadership".
- Sharlin 11mo agoSo what they no longer accept is preprints (or rejects…) It’s of course a pretty big deal given that arXiv is all about preprints. And an accepted journal paper presumably cannot be submitted to arXiv anyway unless it’s an open journal.
- jvanderbot 11mo agoFor position (opinion) or review (summarizing state of art and often laden with opinions on categories and future directions). LLMs would be happy to generate both these because they require zero technical contributions, working code, validated results, etc.
- Sharlin 11mo agoRight, good clarification.
- naasking 11mo agoSo what? People are experimenting with novel tools for review and publication. These restrictions are dumb, people can just ignore reviews and position papers if they start proving to be less useful, and the good ones will eventually spread through word of mouth, just like arxiv has always worked.
- me_again 11mo agoArXiv has always had a moderation step. The moderators are unable to keep up with the volume of submissions. Accepting these reviews without moderation would be a change to current process, not "just like arXiv has always worked"
- naasking 11mo agoSetting aside the wisdom of moderation, instead of banning AI, use it to accelerate review.
- 11mo ago
- amelius 11mo agoMaybe it's time for a reputation system. E.g. every author publishes a public PGP key along with their work. Not sure about the details but this is about CS, so I'm sure they will figure something out.
- jvanderbot 11mo agoTheir name, orcid, and email isn't enough?
- gcr 11mo agoYou can’t get an arXiv account without a referral anyway. Edit: For clarification I’m agreeing with OP
- deleted 11mo ago[deleted]
- hiddencost 11mo agoNot quite true. If you've got an email associated with a known organization you can submit. Which includes some very large ones like @google.com
- mindcrime 11mo agoYou can create an arXiv.org account with basically any email address whatsoever[0], with no referral. What you can't necessarily do is upload papers to arXiv without an "endorsement"[1]. Some accounts are given automatic endorsements for some domains (eg, math, cs, physics, etc) depending on the email address and other factors. Loosely speaking, the "received wisdom" has generally been that if you have a .edu address, you can probably publish fairly freely. But my understanding is that the rules are a little more nuanced than that. And I think there are other, non .edu domains, where you will also get auto-endorsed. But they don't publish a list of such things for obvious reasons. [0]: Unless things have changed since I created my account, which was originally created with my personal email address. That was quite some time ago, so I guess it's possible changes have happened that I'm not aware of. [1]: https://info.arxiv.org/help/endorsement.html https://info.arxiv.org/help/endorsement.html
- DalasNoin 11mo agoit's clearly not sutainable to have the main website hosting CS articles not having any reviews or restrictions. (Except for the initial invite system) There were 26k submission in october: https://arxiv.org/stats/monthly_submissions https://arxiv.org/stats/monthly_submissions Asking for a small amount of money would probably help. Issue with requiring peer reviewed journals or conferences is the severe lag, takes a long time and part of the advantage of arxiv was that you could have the paper instantly as a preprint. Also these conferences and journals are also receiving enormous quantities of submissions (29.000 for AAAI) so we are just pushing the problem.
- marcosdumay 11mo agoA small payment is probably better than what they are doing. But we must eventually solve the LLM issue, probably by punishing the people that use them instead of the entire public.
- mottiden 11mo agoI like this idea. A small contribution would be a good filter. Looking at the stats it’s quite crazy. Didn’t know that we could access to this data. Thanks for sharing.
- skopje 11mo agoI think it worked well for metafilter: $1/1euro one-time charge to join. But that's probably worth it to spam Arxiv with junk.
- nickpsecurity 11mo agoI'll add the amount should be enough to cover at least a cursory review. A full review would be better. I just don't want to price out small players. The papers could also be categorized as unreviewed, quick check, fully reviewed, or fully reproduced. They could pay for this to be done or verified. Then, we have a reputational problem to deal with on the reviewer side.
- loglog 11mo agoI don't know about CS, but in mathematics the vast majority of researchers would not have enough funding to pay for a good quality full review of their articles. The peer review system mostly runs on good will.
- arendtio 11mo agoI wonder why they can't facilitate LLMs in the review process (like fighting fire with fire). Are even the best models not capable enough, or are the costs too high?
- efavdb 11mo agoCurious for the state on things here. Can we reliably tell if a text was LLM generated? I just heard of a prof screening assignments for this, but not sure how that would work.
- jvanderbot 11mo agoOf course there are people who will sell you a tool to do this. I sincerely doubt it's any good. But then again they can apparently fingerprint human authors fairly well using statistics from their writing, so what do I know.
- Al-Khwarizmi 11mo agoThere are tools that claim accuracies in the 95%-99% range. This is useless for many actual applications, though. For example, in teaching, you really need to not have false positives at all. The alternative is failing some students because a machine unfairly marked their work as machine-generated. And anyway, those accuracies tend to be measured on 100% human-generated vs. 100% machine-generated texts by a single LLM... good luck with texts that contain a mix of human and LLM contents, mix of contents by several LLMs, or an LLM asked to "mask" the output of another. I think detection is a lost cause.
- arendtio 11mo agoWell, I think it depends on how much effort the 'writer' is going to invest. If the writer simply tells the LLM to write something, you can be fairly certain it can be identified. However, I am not sure if the 'writer' provides extensive style instructions (e.g., earlier works by the same author). Anecdotal: A few weeks ago, I came across a story on HN where many commenters immediately recognized that an LLM had written the article, and the author had actually released his prompts and iterations. So it was not a one-shot prompt but more like 10 iterations, and still, many people saw that an LLM wrote it.
- physarum_salad 11mo agoThe review paper is dead... so this is a good development. Like you can generate these things in a couple of iterations with AI and minor edits. Preprint servers could be dealing with 1000s of review/position papers over short periods of time and then this wastes precious screening work hours. It is a bit different in other fields where interpretations or know-how might be communicated in a review paper format that is otherwise not possible. For example, in biology relating to a new phenomena or function.
- JumpCrisscross 11mo ago> you can generate these things in a couple of iterations with AI The problem is you can’t. Not without careful review of the output. (Certainly not if you’re writing about anything remotely novel and thus useful.) But not everyone knows that, which turns private ignorance into a public review problem.
- physarum_salad 11mo agoAre review papers centred on novel research? I get what you mean ofc but most are really mundane overviews. In good review papers the authors offer novel interpretations/directions but even then it involves a lot of grunt work too.
- awestroke 11mo agoA good review paper is infinitely better than an llm managing to find a few papers and making a summary. A knowledgeable researcher knows which papers are outdated and can make a trustworthy review paper, an LLM can't easily do that yet
- physarum_salad 11mo agoOk I take your point. However, it is possible to generate a middling review paper combining ai generated slop and edits. Maybe we would be tricked by it in certain circumstances. I don't mean to imply these outputs are something I would value reading. I am just arguing in favour of the proposed approach of arXiv.
- deleted 11mo ago[deleted]
- deleted 11mo ago[deleted]
- bob1029 11mo ago> The advent of large language models have made this type of content relatively easy to churn out on demand, and the majority of the review articles we receive are little more than annotated bibliographies, with no substantial discussion of open research issues. I have to agree with their justification. Since "Attention Is All You Need" (2017) I have seen maybe four papers with similar impact in the AI/ML space. The signal to noise ratio is really awful. If I had to pick a semi-related paper published since 2020 that I actually found interesting, it would have to be this one: https://arxiv.org/abs/2406.19108 https://arxiv.org/abs/2406.19108 I cannot think of a close second right now. All of the machine learning papers are pure slop to me now. The last one I looked at had an abstract that was so long it put me to sleep. Many of these papers aren't attempting basic decorum anymore. Mandatory peer review would fix a lot of this. I don't think it is acceptable for the staff at arXiv to have to endure a Sisyphean mountain of LLM shit. They definitely need to push back.
- programjames 11mo agoThis is only for review/position papers, though I agree that pretty much all ML papers for the past 20 years have been slop. I also consider the big names like, "Adam", "Attention", or "Diffusion" slop, because even thought they are powerful and useful, the presentation is so horrible (for the first two) or they contain major mistakes in the justication of why they work (the last two) that they should never have gotten past review without major rewrites.
- an0malous 11mo agoIsn’t the signal to noise problem what journals are supposed to be for? I thought arxiv was supposed to just be a record keeper, to make it easy to share papers and preprints.
- Al-Khwarizmi 11mo agoYou picked the arguably most impactful AI/ML paper of the century so far, no wonder you don't find others with similar impact. Not every paper can be a world-changing breakthrough. Which doesn't mean that more modest papers are noise (although some definitely are). What Kuhn calls "normal science" is also needed for science to work.
- mottiden 11mo agoI understand their reasoning, but it’s terrible for the CS community not being able to access pre-prints. I hope that a solution can be found.
- sfpotter 11mo agoPlease, read the title and the article carefully. That isn't what's happening.
- swiftcoder 11mo agoIt doesn't apply CS papers in general - only opinion pieces and surveys of existing papers. i.e. it only bans preprints for papers that contribute nothing new.
- ants_everywhere 11mo agoI'm not sure this is the right way to handle it (I don't know what is) but arXiv.org has suffered from poor quality self-promotion papers in CS for a long time now. Years before llms.
- jvanderbot 11mo agoHow precisely does it "suffer" though? It's basically a way to disseminate results but carries no journalistic prestige in itself. It's a fun place to look now and then for new results, but just reading the "front page" of a category has always been a Caveat Emptor situation.
- JumpCrisscross 11mo ago> but carries no journalistic prestige Beyond hosting cost, there is some prestige to seeing an arXiv link versus rando blog post despite both having about the same hurdle to publishing.
- tempay 11mo agoThis isn’t the case in some other fields.
- ants_everywhere 11mo agoBecause a large number of "preprints" that are really blog posts or advertisements for startup greatly increase the noise. The idea is the site is for academic preprints. Academia has a long history of circulating preprints or manuscripts before the work is finished. There are many reasons for this, the primary one is that scientific and mathematical papers are often in the works for years before they get officially published. Preprints allow other academics in the know to be up to date on current results. If the service is used heavily by non-academics to lend an aura of credibility to any kind of white paper then the service is less usable for its intended purpose. It's similar to the use of question/answer sites like Quora to write blog posts and ads under questions like "Why is Foobar brand soap the right soap for your family?"
- exasperaited 11mo agoThe Tragedy of the Commons, updated for LLMs. Part #975 in a continuing series. These things will ruin everything good, and that is before we even start talking about audio or video.
- hoistbypetard 11mo agoSpammers ruin everything. This gives the spammers a force multiplier.
- exasperaited 11mo ago> This gives the spammers a force multiplier. It is also turning people into spammers because it makes bluffers feel like experts. ChatGPT is so revealing about a person's character.
- kibwen 11mo agoPart #975, but that's only because we overflowed the 64-bit counter. Again.
- iberator 11mo agoSimple solution: criminalize posting AI generated publications IF NOT DISCLOSED CLEARLY. Lets say 50000€ fine, or 1 year in prison. :)
- deltaburnt 11mo agoLiterally everything will say AI generated to avoid potential liability. You'll have a "known to the state of California to cause cancer" situation.
- tasuki 11mo agoWould you like to have to prove your comment wasn't written by an AI or would you rather go to prison?
- currymj 11mo agoi would like to understand what people get, or think they get, out of putting a completely AI-generated survey paper on arXiv. Even if AI writes the paper for you, it's still kind of a pain in the ass to go through the submission process, get the LaTeX to compile on their servers, etc., there is a small cost to you. Why do this?
- unethical_ban 11mo agoPresumably a sense of accomplishment to brandish with family and less informed employers.
- xeromal 11mo agoYup, 100% going on a linked in profile
- swiftcoder 11mo agoGaming the h-index has been a thing for a long time in circles where people take note of such things. There are academics who attach their name to every paper that goes through their department (even if they contributed nothing), there are those who employ a mountain of grad students to speed run publishing junk papers... and now with LLMs, one can do it even faster!
- ec109685 11mo agoPublished papers are part of the EB-1 visa rubric so huge value in getting your content into these indexes: "One specific criterion is the ‘authorship of scholarly articles in professional or major trade publications or other major media’. The quality and reputation of the publication outlet (e.g., impact factor of a journal, editorial review process) are important factors in the evaluation”
- Tunabrain 11mo agoIs arXiv a major trade publication? I've never seen arXiv papers counted towards your publications anywhere that the number of your publications are used as a metric. Is USCIS different?
- naveen99 11mo agoIsn’t github the normal way of publishing now for cs ?
- zackmorris 11mo agoI always figured if I wrote a paper, the peer review would be public scrutiny. As in, it would have revolutionary (as opposed to evolutionary) innovations that disrupt the status quo. I don't see how blocking that kind of paper from arXiv helps hacker culture in any way, so I oppose their decision. They should solve the real problem of obtaining more funding and volunteers so that they can take on the increased volume of submissions. Especially now that AI's here and we can all be 3 times as productive for the same effort.
- tasuki 11mo agoThat paper wouldn't be blocked. Have you read the thing?
- zackmorris 11mo agoBefore being considered for submission to arXiv’s CS category, review articles and position papers must now be accepted at a journal or a conference and complete successful peer review. Huh, I guess it's only a subset of papers, not all of them. My brain doesn't work that way, because I don't like assigning custom rules for special cases (edit: because I usually view that as a form of discrimination). So sometimes I have a blind spot around the realities of a problem that someone is facing, that don't have much to do with its idealization. What I mean is, I don't know that it's up to arXiv to determine what a "review article and position paper" is. Because of that, they must let all papers through, or have all papers face the same review standards. When I see someone getting their fingers into something, like muddying/dithering concepts, shifting focus to something other than the crux of an argument (or using bad faith arguments, etc), I view it as corruption. It's a means for minority forces to insert their will over the majority. In this case, by potentially blocking meaningful work from reaching the public eye on a technicality. So I admit that I was wrong to jump to conclusions. But I don't know that I was wrong in principle or spirit.
- habinero 11mo ago> What I mean is, I don't know that it's up to arXiv to determine what a "review article and position paper" is. Those are terms of art, not arbitrary categories. They didn't make them up.
- ninetyninenine 11mo agoDidn’t realize LLMs were restricted to only CS topics. Don’t understand why it restricted one category when the problem spans multiple categories.
- deleted 11mo ago[deleted]
- habinero 11mo agoIf you read through the papers, you'll realize the actual problem is blatant abuse and reputation hacking. So many "research papers" by "AI companies" that are blog posts or marketing dressed up as research. They contribute nothing and exist so the dudes running the company can point to all their "published research".
- internetguy 11mo agoThis should honestly have been implemented a long time ago. Much of academia is pressured to churn out papers month after month as academia is prioritizing volume over quality or impact.
- an0malous 11mo agoWhy not just reject papers authored by LLMs and ban accounts that are caught? arXiv’s management has become really questionable lately, it’s like they’re trying to become a prestigious journal and are becoming the problem they were trying to solve in the first place
- catlifeonmars 11mo agoIt’s articles (not papers) _about_ LLMs that are the problem, not papers written _by_ LLMs (although I imagine they are not mutually exclusive). Title is ambiguous.
- dabber 11mo ago> It’s articles (not papers) _about_ LLMs that are the problem, not papers written _by_ LLMs No, not really. From the blog post: > In the past few years, arXiv has been flooded with papers. Generative AI / large language models have added to this flood by making papers – especially papers not introducing new research results – fast and easy to write. While categories across arXiv have all seen a major increase in submissions, it’s particularly pronounced in arXiv’s CS category. > [...] > Fast forward to present day – submissions to arXiv in general have risen dramatically, and we now receive hundreds of review articles every month. The advent of large language models have made this type of content relatively easy to churn out on demand, and the majority of the review articles we receive are little more than annotated bibliographies, with no substantial discussion of open research issues.
- tarruda 11mo ago> Why not just reject papers authored by LLMs and ban accounts that are caught? Are you saying that there's an automated method for reliably verifying that something was created by an LLM?
- an0malous 11mo agoIf there wasn’t, then how do they know LLMs are the problem?
- 11mo ago
- efitz 11mo agoThere is a general problem with rewarding people for the volume of stuff they create, rather than the quality. If you incentivize researchers to publish papers, individuals will find ways to game the system, meeting the minimum quality bar, while taking the least effort to create the most papers and thereby receive the greatest reward. Similarly, if you reward content creators based on views, you will get view maximization behaviors. If you reward ad placement based on impressions, you will see gaming for impressions. Bad metrics or bad rewards cause bad behavior. We see this over and over because the reward issuers are designing systems to optimize for their upstream metrics. Put differently, the online world is optimized for algorithms, not humans.
- noobermin 11mo agoSure, just as long as we don't blame LLMs. Blame people, bad actors, systems of incentives, the gods, the devils, but never broach the fault of LLMs and their wide spread abuse.
- wvenable 11mo agoWhat would be the point of blaming LLMs? What would that accomplish? What does it even mean to blame LLMs? LLMs are not submitting these papers on their own, people are. As far as I'm concerned, whatever blame exists rests on those people and the system that rewards them.
- jsrozner 11mo agoPerhaps what is meant is "blame the development of LLMs." We don't "blame guns" for shootings, but certainly with reduced access to guns, shootings would be fewer.
- nandomrumber 11mo agoGuns have absolutely nothing to do with access to guns. Guns are entirely inert objects, devoid of either free will nor volition, they have no rights and no responsibilities. LLMs likewise.
- beloch 11mo agoA better policy might be for arXiv to do the following: 1. Require LLM produced papers to be attributed to the relevant LLM and not the person who wrote the prompt. 2. Treat submissions that misrepresent authorship as plagiarism. Remove the article, but leave an entry for it so that there is a clear indication that the author engaged in an act of plagiarism. Review papers are valuable. Writing one is a great way to gain, or deepen, mastery over a field. It forces you to branch out and fully assimilate papers that you may have only skimmed, and then place them in their proper context. Reading quality review papers is also valuable. They're a great way for people new to a field to get up to speed and they can bring things that were missed to the fore, even for veterans of the field. While the current generation of AI does a poor job of judging significance and highlighting what is actually important, they could improve in the future. However, there's no need for arXiv to accept hundreds of review papers written by the same model on the same field, and readers certainly don't want to sift through them all. Clearly marking AI submissions and removing credit from the prompters would adequately future-proof things for when, and if, AI can produce high quality review papers. Clearly marking authors who engage in plagiarism as plagiarists will, hopefully, remove most of the motivation to spam arXiv with AI slop that is misrepresented as the work of humans. My only concern would be for the cost to arXiv of dealing with the inevitable lawsuits. The policy arXiv has chosen is worse for science, but is less likely to get them sued by butt-hurt plagiarists or the very occasional false positive.
- habinero 11mo agoThat doesn't solve the problem they're trying to solve, which is their all-volunteer staff is being flooded with LLM slop and doesn't have the time to artistically moderate. If you want to blame someone, blame all the people LARPing as AI researchers.
- beloch 11mo agoThe majority of these submissions are not from anonymous trolls. They're from identifiable individuals who are trying to game metrics. The threat of boosting their number of plagiarism offences on public record would deter such individuals quite effectively. Meanwhile, banning review articles written by humans would be harmful in many fields. I'm not in CPSC, but I'd hate to see this policy become the norm for all disciplines.
- GMoromisato 11mo agoI suspect that LLMs are better at classifying novel vs junk papers than they are at creating novel papers themselves. If so, I think the solution is obvious. (But I remind myself that all complex problems have a simple solution that is wrong.)
- thatguysaguy 11mo agoVerification via LLM tends to break under quite small optimization pressure. For example I did RL to improve <insert aspect> against one of the sota models from one generation ago, and the (quite weak) learner model found out that it could emit a few nonsense words to get the max score. That's without even being able to backprop through the annotator, and also with me actively trying to avoid reward hacking. If arxiv used an open model for review, it would be trivial for people to insert a few grammatical mistakes which cause them to receive max points.
- HL33tibCe7 11mo ago> I suspect that LLMs are better at classifying novel vs junk papers than they are at creating novel papers themselves. Doubt LLMs are experts in generating junk. And generally terrible at anything novel. Classifying novel vs junk is a much harder problem.
- generationP 11mo agoI have a hunch that most of the slop is not just on CS but specifically about AI. For some reason, a lot of people's first idea when they encounter an LLM is "let's have this LLM write an opinion piece about LLMs", as if they want to test its self-awareness or hack it by self-recursion. And then they get a medley of the learning data, which if they are lucky contains some technical explanations sprinkled in. That said, AI-generated papers have already been spotted in other disciplines besides cs, and some of them are really obvious (arXiv:2508.11634v1 starts with a review of a non-existing paper). I really hope arXiv won't react by narrowing its scope to "novel research only"; in fact there is already AI slop in that category and it is harder to spot for a moderator. ("Peer-reviewed papers only" is mostly equivalent to "go away". Authors post on the arXiv in order to get early feedback, not just to have their paper openly accessible. And most journals at least formally discourage authors from posting their papers on the arXiv.)
- zekrioca 11mo agoTwo perspectives: Either (I) LLMs made survey papers irrelevant, or (II) LLMs killed a useful set of arXiv papers.
- whatpeoplewant 11mo agoGreat move by arXiv—clear standards for reviews and position papers are crucial in fast-moving areas like multi-agent systems and agentic LLMs. Requiring machine-readable metadata (type=review/position, inclusion criteria, benchmark coverage, code/data links) and consistent cross-listing (cs.AI/cs.MA) would help readers and tools filter claims, especially in distributed/parallel agentic AI where evaluation is fragile. A standardized “Survey”/“Position” tag plus a brief reproducibility checklist would set expectations without stifling early ideas.
- whatever1 11mo agoThe number of content generators is now infinite but the number of content reviewers is the same. Sorry folks but we lost.
- jsrozner 11mo agoI had a convo with a senior CS prof at Stanford two years ago. He was excited about LLM use in paper writing to, e.g., "lower barriers" to idk, "historically marginalized groups" and to "help non-native English speakers produce coherent text". Etc, etc - all the normal tech folk gobbledygook, which tends to forecast great advantage with minimal cost...and then turn out to be wildly wrong. There are far more ways to produce expensive noise with LLMs than signal. Most non-psychopathic humans tend to want to produce veridical statements. (Except salespeople, who have basically undergone forced sociopathy training.) At the point where a human has learned to produce coherent language, he's also learned lots of important things about the world. At the point where a human has learned academic jargon and mathematical nomenclature, she has likely also learned a substantial amount of math. Few people want to learn the syntax of a language with little underlying understanding. Alas, this is not the case with statistical models of papers!
- anupj 11mo ago[dead]
- _jsmh 11mo ago"review articles and position papers must now be accepted at a journal or a conference and complete successful peer review." How will journals or conferences handle AI slop?
- deleted 11mo ago[deleted]
- goldenjm 11mo agoTheir argument in favor of this change seems extremely reasonable and well-explained.
- hamonrye 11mo ago[dead]
- Quizzical4230 11mo agoShameless plug. PaperMatch [1] helps solve this problem (large influx of papers) by running a semantic search on top of abstracts, for all of arXiv. [1]: https://papermatch.me/ https://papermatch.me/
- jruohonen 11mo agoA very weird move. They are now taking a stance on what science is supposed to be. As someone commented, due to the increasing volume, we would actually need and benefit from more reviews -- with a fixed cycle preferably, and I do not mean LLM slop but SLRs. And in contrary to someone's post, it is actually nice to read things from the industry, and I would actually want that more. And not only are they taking a stance on science but they have also this allegation: "Please note: the review conducted at conference workshops generally does not meet the same standard of rigor of traditional peer review and is not enough to have your review article or position paper accepted to arXiv." In fact -- and supposedly related to the peer review crisis, the situation is exactly the opposite. That is, reviews are usually today of much higher quality at specialized workshops organized by experts in a particular, often niche area. Maybe arXiv people should visit PubPeer once in a while to see what kind of fraud is going on with conferences (i.e., not workshops and usually not review papers) and their proceedings published by all notable CS publishers? The same goes for journals.
- jruohonen 11mo agoOne thing I forgot to speculate: a position paper on DEI and Cornell University...
- kittikitti 11mo agoIn my experience, arXiv is not a preprint platform. It's a strange gatekeeper of science and should be avoided altogether. They have their favorites which they deem as "high quality" and everything else gets rejected. I am eagerly awaiting for people to dismiss arXiv altogether.
- whatpeoplewant 11mo ago[flagged]