7 ms·
It looks like the status/need-triage label was removed
- minimaxir 8mo agoIt's not wrong.
- robertclaus 8mo agoClassic CI bug with a flair of LLM fun! We had something similar creep into our custom merge queue a few weeks back.
- embedding-shape 8mo agoWhat "classic CI bug" makes bots talk with each other forever? Been doing CI for as long as I've been a professional developer, and not even once I've had that issue. I've made "reply bots" before, bunch of times, first time on IRC, and pretty much the second or third step is "Huh, probably this shouldn't be able to reply to itself, then it'll get stuck in a loop". But that's hardly a "classic CI bug", so don't think that is what you're referring to here right?
- btown 8mo agoIf you’re making a bot in which there will be many sub-behaviors, it can be tempting to say “each sub-behavior should do whatever checks it needs, including basic checks for self-reply.” And there lie dragons, because whether a tired or junior or (now) not-even-human engineer is writing new sub-behavior, it’s easy to assume that footguns either don’t exist or are prevented a layer up. There’s nothing more classic than that.
- embedding-shape 8mo agoI'm kind of understanding, I think, but not fully. Regardless of how you structure this bot, there will be one entrypoint for the webhooks/callbacks, right? Even if there is sub-behaviours, the incoming event is passing through something, or are we talking about "sub-bots" here that are completely independent and use different GitHub users and so on? Otherwise I still don't see how you'd end up with your own bot getting stuck in a loop replying to itself, but maybe I'm misunderstanding how others are building these sort of bots.
- pixl97 8mo agoAll I can think of, and actually have seen is 1. Bot run a series of steps A through Z. 2. Step X is calling an external system that runs its own series of steps. 3. Some potential outcomes of said external system is if it detects some potential outcomes (errors, failed tests, whatever) is it kicks back an automated process that runs back through the bot/system where said system makes the same mistake again without awareness it's caught in a loop.
- matsemann 8mo ago1. Set up a bot that runs on every new comment on a PR 2. The bot comments something on that PR Doesn't have to be more advanced than this to get an infinite loop if you don't build anything where it ignores comments from itself or similar.
- embedding-shape 8mo agoPreviously: > pretty much the second or third step is "Huh, probably this shouldn't be able to reply to itself, then it'll get stuck in a loop". But that's hardly a "classic CI bug",
- matsemann 8mo agoIf I've previously misunderstood your point, copy pasting it doesn't clear anything up, no..? I don't see why it's not a "classic CI bug". It's an easy trap to fall into, and I've seen it multiple times. Same with "action that runs on every commit to main to generate a file and push a new commit if the file changes", that suddenly gets stuck in a loop because the generated file contains a comment with the timestamp of creation.
- btown 8mo agoSorry, could have been more clear. Someone sets up a bot with: on a trigger, read the message, determine which "skill" to use out of a set of behaviors, then let that skill handle all the behavior about whether or not to post. Later, someone (or a vibe coding system) rolls out a new skill, or a change to the skill, that omits/removes a self-reply guard, making the assumption that there are guards at the orchestration level. But the orchestration level was depending on the skill to prevent self-replies. The new code passes linters and unit tests, but the unit tests don't actually mimic a thread re-triggering the whole system on the self-posting. New code gets yolo-pushed into production. Chaos ensues.
- Hamuko 8mo agoYeah, a bot replying to itself is pretty poor design. It's one of the first things you do even with toy bots. You can even hardcode knowing itself, since usually you have an unchanging ID. A much more common problem is if someone deploys another bot, which will lead your bot into having an endless back-and-forth with it.
- embedding-shape 8mo ago> A much more common problem is if someone deploys another bot, which will lead your bot into having an endless back-and-forth with it. This I'd understand, bit trickier since you're basically end up with a problem typical of distributed systems. But one bot? One identity? One GitHub user? Seems really strange to miss something like that, as you say, it's one of the earlier things you tend to try when creating bots for chats and alike.
- matsemann 8mo agoBeing one of the earlier things to catch is what makes it a classic.
- embedding-shape 8mo agoIt's easy to miss, but in the middle of the page: > 4609 remaining items Seems gemini-cli and gemini-cli didn't understand who themselves were, so they though someone else added/removed the label, which it tried to correct, which the other then tried to correct, which the other... Considering that that repository has what seems like ~10 longer term contributors, who probably get email notifications, together with a bunch of other people who get notifications about it, wonder how many emails were sent out because of this? If we just assume ten people get the emails, it's already 46K emails going out in under 24 hours... Also, who pays for the inference of this gemini-cli? Clicking the "user" links to https://github.com/apps/gemini-cli https://github.com/apps/gemini-cli, and it has a random GitHub user under "Developer", doesn't seem like it's a official Google project, so did someone pay for all of these inference calls here? That'd be a pretty sucky bill to pay...
- TACD 8mo agoThis isn't the first time it's happened, either. It's a pretty frequently recurring issue, in fact: https://github.com/google-gemini/gemini-cli/issues/16723 https://github.com/google-gemini/gemini-cli/issues/16723 https://github.com/google-gemini/gemini-cli/issues/16725 https://github.com/google-gemini/gemini-cli/issues/16725 https://github.com/google-gemini/gemini-cli/issues/16732 https://github.com/google-gemini/gemini-cli/issues/16732 https://github.com/google-gemini/gemini-cli/issues/16734 https://github.com/google-gemini/gemini-cli/issues/16734
- embedding-shape 8mo agoAll opened the 15th of January though, same as the instance linked in the submission. Seems maybe more accurate to say "widespread issue" rather than "frequent issue", as it seems to only have happened at one occasion, but it had time to spam many issues on that day.
- 293736729129 8mo ago[dead]
- eviks 8mo ago
- alwa 8mo agoThis issue seems to involve Gemini-cli[bot] squabbling with itself, adding and removing the label from the issue (leaving contradictory explanation comments to itself each time) for a good 4,600 rounds
- add-sub-mul-div 8mo agoI don't lament the lack of flying cars because they don't seem practical, but I am disappointed that the future turned out to be this stupid.
- ryandrake 8mo agoA similar issue made HN last week, same repo, where an AI bot was having the same kind of argument with itself over and over on an issue. Someone mentioned: This sort of thing is why RAM is 800 bucks now.
- omoikane 8mo agoThis is the thread: https://news.ycombinator.com/item?id=46636291 https://news.ycombinator.com/item?id=46636291
- amiga386 8mo agoProject admins setting up automation: https://youtu.be/B4M-54cEduo?t=102 https://youtu.be/B4M-54cEduo?t=102 The automation: https://youtu.be/GFiWEjCedzY?t=51 https://youtu.be/GFiWEjCedzY?t=51
- PyWoody 8mo agoHeh. This reminds me of the time when our newly hired "Salesforce Expert" improved our support queue: Every time Support received a new email, a ticket in Salesforce would be created and assigned to Support Every time Support was assigned a new ticket, Salesforce would send a notification email The worst part is he wouldn't admit to the mistake and it took us forever to find where he buried the rule.
- pinkmuffinere 8mo agolol, that's amazing. Things like this make me both angry (how could they be so dumb!), and empathetic (what is the rest of their life like?)
- pousada 8mo agoI only used salesforce once (was “forced” to use it haha) and it was mind boggling how anyone would ever want to use it or even become an expert in using it. I’d rather track everything in a giant excel tyvm
- embedding-shape 8mo ago> it was mind boggling how anyone would ever want to use it or even become an expert in using it. As in a lot of cases, the answer is money. If you have expertise in Salesforce, you can get paid a lot, especially if the company you contract/freelance for is in an "emergency" which, because they use Salesforce, they'll eventually be. As long as you get the foot in the door, you'll have a steady stream of easy money. It fucking sucks though, the entire ecosystem, not for the weak of heart.
- GuinansEyebrows 8mo agonobody who actually uses salesforce for daily work chose it. it's sold directly to CIO/CTOs as a one-stop shop for CRM, ticketing, reports and biz dev, who may occasionally use it for reporting (but more often get their staff to provide the reports directly to them). everybody stuck having to use it to actually track work just has to suffer with it.
- keriati1 8mo agoToday github labels, tomorrow paperclips?
- a-dub 8mo agoin the old days one would add and check for a loop detection token when loops like this could be driven by external systems... i wonder if today it would be as simple as adding "ensure you don't get stuck in any loops" to a prompt. fwiw. doesn't look like gemini at all, the responses are perfectly canned... maybe just good old fashioned ci rules.
- vjekm 8mo agoI also start all of my prompts with "solve the halting problem."
- fragmede 8mo agoClang manages to have way more useful error messages despite not solving three halting problem. You don't need to solve the halting problem to have caught this problem. Even if you don't solve it for the general case of the halting problem, solving it here for a levels deep and then collapsing the levels would have stopped this problem in its tracks. Sure, someone could just come in and cause the bug at N+1 levels deep because you've only solved it at N, but you can write different tests to mitigate that problem in practice, despite not having infinity RAM *2+1 to solve the general case of the halting problem. Hilariously, the halting problem has been written in enough of the LLM training data that it can identify some cases where the code won't terminate.
- Night_Thastus 8mo agoIt's a language model. It doesn't know what a loop is, or have any awareness of that the content it's replying to may be made by itself - as it has no sense of 'self'.
- supernes 8mo agoFinally an example of AI doing something useful. Imagine having to add and remove all those tags 4500+ times by hand!
- heliumtera 8mo agoProfessionals GitHub labels adder-remover became obsolete. AGI practicality achieved
- Phui3ferubus 8mo ago> 4610 remaining items Normally I would complain about people spamming in GitHub issues but I don't think it will matter this time
- heliumtera 8mo agoSo much productivity accomplished here! Those are numbers management loves to see. gemini-cli did much more work in this PR then the author himself.
- armchairhacker 8mo agoIronically this is type of issue is common in pre-LLM (rules-based) AI. Given that the back-and-forth messages are the same, I suspect they're generated by a small script, not an LLM. But I wouldn't be surprised if the script was created mostly or entirely by an LLM.
- Elfener 8mo agoMaybe I'm missing something, but this seems to be an issue report claiming to be a PR? Where's the patch? Edit: there's actually a PR, but this is one of those repos where for some reason, they require every PR to have an associated issue. And in this case, they aren't even linked...
- venturecruelty 8mo ago[dead]
- jayd16 8mo agoI think the real irony is an LLM trying to enforce permissions at all. Why is it doing that? If the tag exists, the user had the permission to create it, no?
- throwaway173738 8mo agoI’m guessing there’s no permissions around labeling an issue in Github.
- mise_en_place 8mo agoNow that's what I call job security.
- abathologist 8mo agoThis will soon be happening with our parents' social security checks, our friend's cancer treatment plan, our international flights logistics, our ISPs routing configurations, ... Fun times are coming.
- meisel 8mo agoTo be clear, is AI actually at play here, aside from the fact that the repo is for Gemini? It just looks like two simple rules that interact poorly, that we could've seen in 2015.
- tuetuopay 8mo agoWell, it's even more ironic as AI in general is touted as smart. I'd fully expect such bots to notice they're in a loop and one to throw the towel. Still a long way to AGI. And to AI for that matter.
- bdmorgan 8mo agoScript author here :-) This was due to two different GitHub Action workflows: (Workflow 1): Remove the need-triage label under certain conditions. (Workflow 2): If anyone outside a project maintainer removes a label, re-add it with a friendly message explaining why. Submitted those at like 10 or 11 pm and went to sleep. Woke up to all issues that got changed overnight with dozens, hundreds, or thousands of these messages. Cause: Workflow 2 should have checked for project maintainers but also other bots and automation that might also be clearing labels. It got fixed immediately once we realized the issue.
- doodlesdev 8mo ago> Submitted those at like 10 or 11 pm and went to sleep. That's a classic :) Hopefully this hasn't caused any real harm. At least it sure did give me a good laugh when I first saw it.
- fragmede 8mo agoIn corporate, that's pushed to prod and then got on an international flight on a Friday afternoon.
- storystarling 8mo agoI learned the hard way to always implement a circuit breaker for event-driven triggers like this. We use a simple Redis counter with a short TTL to rate limit execution and fail fast if it detects a loop. It is standard practice in backend queues like Celery but easy to overlook in CI configurations.