8 ms·
PR spam today looks like email spam in the early 2000s
- deleted 3mo ago[deleted]
- Retr0id 3mo agoMaybe we should cut out the middle-man and make it easy for people to donate token credits to open-source projects, and let the maintainers decide how to use them.
- bluefirebrand 3mo agoUnfortunately "I donated money/tokens to open source" doesn't land interviews as well as "I'm a big contributor to open source" People spamming Open Source repos with AI PRs aren't trying to help Open Source, they're trying to build a brand, some kind of credible online presence with their username on it, or whatever else. It's purely selfish and completely opposite to the spirit of Open Software imo
- bitmasher9 3mo agoThis is the most uncharitable outlook on the increase of PRs. It may be true for some contributors, but any company reviewing their GitHub will see that the code is largely spam. I think most AI generated code is people that want to help the project, but maybe aren’t familiar with the standards and norms.
- bluefirebrand 3mo agoI don't have much charity in my heart for people using AI to spam every corner of the public internet with slop
- Planktonne 3mo agoIf their AI model is good enough to make PRs, it's good enough for them to ask it about standards and norms. They can't be both massively enabled and helplessly unaware.
- ffaccount2 3mo ago>People spamming Open Source repos with AI PRs aren't trying to help Open Source, they're trying to build a brand I am certain many of them honestly believe that they are doing the right thing and that they are helping. After all hey, they implemented a feature or fixed a bug for the community! It's a grim worldview if you think they are all just selfish.
- parliament32 3mo agoThey're stuck in this idea that somehow they're better at prompting the slop generator than anyone else, therefore they're helpful and people definitely want their output merged in to these various projects. They will have trouble understanding that their personal contribution to the whole process is somewhere between negligible and harmful, and simply donating those tokens to a maintainer who is actually aware of how the codebase works and where all the skeletons are is a much better proposition.
- Larrikin 3mo agoI would argue this is naive and there's very little evidence to support this opinion other than just wishing it was true. It may happen on smaller projects with few users but not in meaningful large projects.
- janalsncm 3mo ago> there's very little evidence to support this opinion other than just wishing it was true Building a brand doesn’t require submitting to someone else’s open source project. You can do the same thing by creating your own OSS project. For a lot of them it’s probably a little of column A and a little of column B. If people are submitting in their real name it’s more likely they’re building a brand. I also think it’s possible for someone to genuinely think they are helping without trying to build reputation.
- Sharlin 3mo agoOh but you see, own OSS projects are not worth much unless they got stars. Anyone can now fill their GitHub space with a hundred vibecoded projects in an afternoon, it's worth nothing unless it comes with social proof.
- parliament32 3mo agoFor now. Give it another half year and "I contribute to open source" will carry the same weight as "I donate to charity" ie nobody cares because any idiot can do it. I wonder how long it'll take before "I don't use LLMs for coding" carries weight.
- sureglymop 3mo agoInterestingly then, those contributions are also not a measurement of the candidates abilities but mostly of the AI models. I wonder if hiring adjusts to that but I doubt it. It might only push it even more towards "marketing matters most" instead of actual ability.
- stackghost 3mo ago>I wonder if hiring adjusts to that but I doubt it Tech hiring/interviews have almost nothing to do with assessing the candidates' ability to do the job.
- janalsncm 3mo agoThere are so many leetcode questions where solving it requires knowing some trick. Part of the trap for SWEs is that once you know the trick you feel smarter, but it really has nothing to do with software engineering. Now that Claude is the best leetcoder in the world it would be great if companies which intend to hire humans would reconsider asking such dumb questions.
- slaymaker1907 3mo agoI've personally started focusing a lot more on code quality and communication skills over correctness of solving some leetcode problem. If I could get the infrastructure in place for it in the interview, I would have candidates generate something via AI and watch their process for that (how do they evaluate a plan, how do they review the code, etc.).
- janalsncm 3mo agoForce them to use a bad LLM and clean up the code?
- toss1 3mo agoA fine example of Goodhart's law: "When a measurement becomes a target, it ceases to be a good measurement." Measuring open source contributions as a way to judge prospective employees used to be a good measurement. Of course, prospective employees started to not only contribute to OS projects because it was good, but to make sure their contributions were high and noticeable — contributing not for the good of the project but for their own good, and now with amplification of AI 'contributions'. So, measuring contributions to open source projects is now approximately worthless for evaluating prospective employees.
- elif 3mo agoMaybe I'm optimistic or not typical but in my experience people submit random PR to open source projects because they really want the project to do xyz for their own project/reasons, and the project doesn't do xyz. And the PR is considered "spam" because the maintainer doesn't see xyz as part of his needs or his vision for the project.
- Sharlin 3mo agoBeing able to donate tokens won't help with that, unless the project maintainers also want the project to do xyz.
- pavel_lishin 3mo agoLike this? https://news.ycombinator.com/item?id=48621645 https://news.ycombinator.com/item?id=48621645
- jayd16 3mo agoHow about just cash?
- mort96 3mo agoMaybe we should cut out the middle man and make it easy for people to donate money to open-source projects, and let the maintainers decide whether to use them on tokens or hosting or developer salaries or something else.
- pornel 3mo agoSo that's how the sci-fi dystopias end up using "credits" for their money.
- edm0nd 3mo agoas an open source project maintainer, we dont want tokens bro, we want $.
- junon 3mo agoAh yes, the for-profit companies that trained their commercial models off of all our open source code from the last 50 years need more money from us.
- runarberg 3mo agoAI agents who review the slop created by other AI agents is not the answer here. I much prefer a blanket ban on PRs and issues created by AI agents (which is what I personally do for my repos; so far I have closed one[1]). In fact I would love a github alternative which considers AI contributions to be a breach of their terms of use and ban any people who let AI agents loose on their platform. 1: https://github.com/runarberg/markdown-it-math/pull/48#issuecomment-4665209260 https://github.com/runarberg/markdown-it-math/pull/48#issuec...
- margalabargala 3mo agoI tend to disagree. I think the comparison to email spam is apt. The answer to that problem was automated spam filters. Imagine the difficulty you might find interacting with the world if your inbox was set up such that all emails not literally written by a human were auto-deleted. No account recovery, no receipts, etc. Individuals might choose to do that for themselves but it's not the general case answer.
- sigbottle 3mo agoThat's different though - those are services you explicitly agree to and sign up for, be it at checkout, be it at service signup time, be it because you are making a google account on the google platform. For example, a github cicd automerge pipeline is still good.
- thayne 3mo agoAnd automated spam filters are a poor solution. Even after a couple decades the best ones still have both false positives and false negatives at a higher rate than I would like.
- CapsAdmin 3mo agoOne interesting workflow I've seen is that the project maintainer simply rewrites and implements the pull request themselves and closes the PR. LuaJIT has operated this way since 2012, though with a thanks and mention in the commit message. It seems like a good way to filter out people who prioritizes leveling up their github profiles. Something a little bit similar, when I was hosting a social game server we had mods. And players always beg for mod status. At first I tried naming the admin group something weird like sandals, but eventually people would ask if they could be sandals too. What worked best in the end was just hiding it completely making regular players see mods as other regular players. (mods would see who is a mod though) I would also personally never make someone who asks a mod as it's almost always a sign of wanting power for the sake if it. I would instead just passively observe behavior until I trusted the player and make them a mod. I would then tell them that I don't expect them to exercise their power, but would demote if I see abuse of power.
- j2kun 3mo agoIn my main project we added a new requirement that all new contributors meet a maintainer in a non-textual format before their first PR is merged. Seems to work well for a small project.
- idiotsecant 3mo agoWhat an elegantly common sense solution. It's also probably a really good way to make contacts with interesting people.
- boredatoms 3mo agoLike a video/phone call?
- bluGill 3mo agoI'm not sure if AI can do those today, but they probably can in the near future. (probably we will be able to see obvious "that can't be human" for a while longer)
- Chu4eeno 3mo agoIf you (or even your pet LLM) is able to set up v4l-loopback and some convincing realtime image/audio gen I think that's a signal that your PRs might be worth reading.
- idiotsecant 3mo agoThe point at which an AI can convince me in a video call revolving around a complex social interaction like an introduction and discussion of interests that it's human I'm gonna go ahead and let it have the title.
- hnlmorg 3mo agoIt already can and it’s a big problem in recruitment. But for PRs I suspect it isn’t a big concern because this filter is to weed out PR spam from people who want to invest time in the project.
- 3mo ago
- fecal_henge 3mo agoCan I ask what the motive is to create agents to do this? Where is the profit?
- kridsdale1 3mo agoI think there are a lot of “tech schools” overseas that require students to show proof of contribution to open source.
- jimbokun 3mo agoIt would be wonderful if the instructors at those schools built relationships with open source maintainers and the maintainers knew when their students were submitting PRs. Could be used as a teaching experience that many maintainers would be happy to participate in, instead of feeling attacked with random low quality PRs.
- tokioyoyo 3mo agoYou might be underestimating the number of little schools, and computer shops. I can recall even back in 2005, there were HTML shops popping up here and there, in little cities around the world.
- dkdbejwi383 3mo agoOpen source contributions being a great way to learn and to pad out your CV has been considered good advice on all sides of the various seas I’ve lived throughout my career too - it’s not just a dubious code camp thing.
- cheald 3mo agoA robust open source profile is my single favorite hiring profile indicator. However, with the current state of things, if I get a whiff of AI-driven "contribution" it becomes an instant black mark against the candidate.
- giancarlostoro 3mo agoDoes github not have rulesets for who can even try to do a PR? I would lockdown my repositories if I didn't want any PR slop.
- ValdikSS 3mo agoThey do, that's a relatively recent feature: https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/enabling-features-for-your-repository/disabling-pull-requests https://docs.github.com/en/repositories/managing-your-reposi...
- thayne 3mo agoMost open source projects want outside contributions... as long as they are decent quality. The problem is how do you block the AI slop spam without also blocking high quality contributions.
- janalsncm 3mo agoI understand this is a general problem in OSS, but I also hope the irony isn’t lost that this article is specifically complaining about AI slop PRs to the Open Claw repo. If the maintainers are that tired of it, they should update OpenClaw to prevent it from submitting PRs to their repo.
- thayne 3mo agoAnd the fact that this article doesn't acknowledge that irony diminishes my respect for it.
- aniokono 3mo agoWhat are the best solutions to this issue?
- andix 3mo agoI see one big difference: with email it was always about sender reputation based on email servers (IPs), maybe about domains. But never about individual users. It's the organizations running the email server, who make sure users behave. So they don't get blacklisted and lose sending privileges for hundreds or thousands of users. For PRs/issues this is not applicable.
- decimalenough 3mo agoNot necessarily. Orgs exist in GitHub, and it seems reasonable that if the $BIGCORP org limits membership to employees, you can automatically trust all members of that org. Because this way, if one steps out of line, you have both an escalation path (contact admins) and a stick (revoke trust in entire org).
- VortexLain 3mo agoAllowing contributions only from big tech companies sounds ideologically questionable from free/libre software movement perspective, and it emboldens decisions which go against the user's interests, such as removing manifestv2 in Chromium.
- janalsncm 3mo agoIt doesn’t have to just be companies. It could be some kind of guild with standards and application criteria. That group could vet members and kick them out for posting slop code.
- 3mo ago
- guidoiaquinti 3mo agoGitHub just recently added configurable PR limits for maintainers to help partially address this problem: https://github.blog/open-source/maintainers/how-pull-request-limits-are-cutting-down-the-noise/ https://github.blog/open-source/maintainers/how-pull-request...
- IshKebab 3mo agoI would not be at all surprised if Github adds a first party reputation system. It would be a clever way to increase network effects - imagine if you host on Codeberg you're inundated by AI PRs but on Github you can easily filter them out. I can't see those pull request limits working very well. It's like trying to filter email spam by just rate limiting people. It's going to be annoying for the people you actually want to talk to, and you're still going to get at least 1 spam message from every spammer out there.
- newswasboring 3mo agoIf we want to keep it objective, one metric can already be calculated based on the user history of the submitter. The spammers profile will be full of unmerged or abandoned prs. Just based on those statistics beginners might be close to zero rating but spammers would be negative. Unless I totally missed that people are also making new accounts of each PR.
- xbmcuser 3mo agoPrs are too pad cv's or novice users wanting something and vibe coding it themselves. Jellyfin player repos see a lot these kinds of prs so unlikely to be new users
- okigan 3mo agoOr creating repos that will merge their PR(s).
- csomar 3mo agoCreate lots of fake repos/prs to improve your ratio?
- othmanosx 3mo ago[flagged]
- benj111 3mo agoWait. So to combat AI spam there's AI agents to prevent it? Why can the anti spam agents not just do the work directly???
- deleted 3mo ago[deleted]
- alexpotato 3mo agoIf anyone is interested in what it was like fighting spam in the early 2000s, I worked for a company that captured spam, analyzed it and then passed the analysis s on to the law firms of the big email providers for targeting under CAN-SPAM. Twitter thread about it below but happy to do a AMA here. https://x.com/alexpotato/status/1208948480867127296?s=20 https://x.com/alexpotato/status/1208948480867127296?s=20
- rapind 3mo agoIt's the same scaling issue we've had since the advent of the internet, and why spam and social media became such a dumpster fire. There are many things in life that are perfectly fine when uncommon / rare, but are disastrous when done cheaply at scale.
- Chu4eeno 3mo agoIronically one of the first recognizable spam campaigns was perpetrated by lawyers: https://en.wikipedia.org/wiki/Laurence_Canter_and_Martha_Siegel https://en.wikipedia.org/wiki/Laurence_Canter_and_Martha_Sie...
- alexpotato 3mo agoOn the flip side, the lawyers that represented the big tech firms at the time were some of the most impressive people I've ever met. You could speak to them as a peer when it came to technical issues or system architecture AND they were experts in technology law. Especially impressive given that anti-spam was still in it's infancy and rapidly evolving.
- Chu4eeno 3mo agoYeah, I didn't mean to slight lawyers in general, some of my best friends are lawyers (one of them even convinced me to switch to Signal before it was cool).
- davidwritesbugs 3mo agoWhen I was a lawyer my selling point was exactly that I knew tech deeply, coming from a dev background, and was current as I read Slashdot daily compiled my own Linux kernels etc. It allows punters trust you & is a rainmaking skill.
- elzbardico 3mo agoI remember that on the not so early days of the internet around 1993, I managed to exchange emails with pretty much important people, known professionals and even got responses to my questions. It looked like a very very small world. Then, came the spam. I really hate the marketing people mindset. It fucks everything that is nice.
- splendidz 3mo ago[dead]
- mrbonner 3mo agoFun fact: it is spam filtering application that makes Paul Graham famous (and rich)
- dang 3mo agoPG was certainly known for that (e.g. https://www.joelonsoftware.com/2003/11/22/22-2/ https://www.joelonsoftware.com/2003/11/22/22-2/) but I don't think it made him rich. Selling Viaweb did.
- cat_plus_plus 3mo agoIt's not at all like e-mail spam. Vast majority of contributors made a change useful for themselves that they wish to share with others. It's better to think of this as an influx of new programmers or existing programmers picking up new domains. They can be taught to use coding agents better and are likely to stick with projects that facilitate this rather than shutting them out. Maybe it's best for everyone. Let Linux kernel be super locked down to l33t contributors only and let alternative OSes that nobody paid attention to before gain new developers.
- gmziven 3mo ago[flagged]
- haneul 3mo agoI didnt think about it this way, but it would be helpful to have infra around reputation in the same way email has them. And, it would be nice to have unsubscribe lists in the uBlock Origin style.
- DeepYogurt 3mo agoWhy though? Is everyone karma farming? What's the motivator here?
- ilsubyeega 3mo agoto get reputation, may benefits from huge companies such like github(via copilot) etc.
- DeepYogurt 3mo agoLike what? Do you get tokens for "helping open source" or something?
- sph 3mo agoThey honestly think they're doing the world a favour, and their AI slop is helpful. It is clear by now that there are a lot of people with not a great deal of self-awareness.
- usernametaken29 3mo agoWhen the title said PR spam my first thought was the massive amount of scam posts released by Antrophic, OpenAI and a gazillion AI tools that solve all your problems (supposedly). Much to my surprise this was also an ad for an AI tool. Lowkey disappointing
- imrehg 3mo agoI value Open Source very highly, and enjoy contributing. Fix a bug here, add a small feature there, most OSS projects have low hanging opportunities. And these days, to "switch off" from my at-work code, try to do more OSS contribution as well. What I see instead, really, is that most projects no longer, or very rarely look at any contribution, and e.g. any issue + PR/MR combo I make, has a much higher chance of never being looked at and some bot just closes it. Even though the rest of the project might be actually quite active. It takes some getting used to, apparently being filed together with the "noise", when I try to go out of my way to be as much "signal" as possible. But well, if I really wanted to, I can just run my own changes locally, that's the beauty of OSS, but I hope we can get to some more balanced place over time (being forever the optimist).
- uberman 3mo agoI have the same pattern and same observation. No one wants a PR any more as far as I can see. In some sense, this has always been the case. People are happy to do their own thing but not so keen on integration with other people's code. Now though it is on a different level. I can see maintainers are being overwhelmed by AI driven PRs but if we filter our new features and concentrate just on bug fixes, does it really matter where a PR comes from if it fixes a bug? I do work on a very complicated agent based simulation. The data shaping and loading is all open source python. There are dozens of long standing bugs that prevent the simulation from loading some of the data correctly. I used to send PRs but they were always ignored so I gave up. Now, when there is a new release I need to spend a day reviewing the new code to see what patches I need to re-apply.
- mikepurvis 3mo ago> fixes a bug Few changes are as cut and dried as fixing a bug with zero side effects or change in behaviour or need to consider future support or architectural plans. As a maintainer, I'm always happy to quickly merge something that's like "I hit this corner case, here's a tightly-scoped change that catches and fixes it" but in reality not a lot of changes actually looked like that.
- zackmorris 3mo ago
- aryaabyte 3mo agowhich is why I found this https://pr-captcha.aryaabyte.workers.dev/ https://pr-captcha.aryaabyte.workers.dev/ pr captcha
- K0balt 3mo agoIt’s amazing how even programmers can’t resist defecating on their own lawn.
- tancop 3mo agotangled has a new web of trust system where you can denounce or vouch for a user and all your friends see it. it decays over time and the whole thing is based on atproto so its easy to set up automated community lists on top.