8 ms·

When I reject AI code even if it works

237 points by vnbrs 3mo ago

codelong888 3mo ago
[flagged]
deleted 3mo ago
[deleted]
_wire_ 3mo ago
"Even if it works?" How do you verify that it works?
- p1024k 3mo ago
  According to the author's intention, it is the code that he cannot understand or control. Even if the solution provided by the AI works, he will not adopt it. This is unless he can understand or control it. This should be an assumption. However, if AI provides a solution, as the person using AI, one should conduct research before making a decision. This is not in conflict with or hindered by the use of the ideas provided by AI.
  - andyfilms1 3mo ago
    I will say--as someone who has fielded late night troubleshooting calls--I totally understand OP's point of view. It's reasonable to expect that you will be able to answer questions about something that you ship, or brainstorm ways to solve a problem a customer is encountering while using something you provided them. The obvious counterargument is "well, just ask the AI for those answers," but the AI lacks the context and experience that you have. Sometimes, genuinely, the user really is just "holding it wrong," but none of the current AI models would ever admit that, and you'd spend hours trying to solve an unsolvable problem.
  - Grombobulous 3mo ago
    I think this policy is probably more prescriptive than I would go with myself. I like to think of my risk tolerance first to help make that determination. For example, I use a vibecoded internal tool written in Go. I don’t even know how to write Go. Haven’t read a single line of the code. I just wanted to move from bash scripts to using cloud SDKs for performance reasons. But the internal tool is a convenience tool, and you can do everything it does using alternative methods. So if it break, there is no real negative impact besides personal convenience of anyone using it. There’s some documentation on how to do everything manually if needed. Here’s another example: you’re making a static website. No JavaScript, no interactivity. Truly, what could go wrong? And while I do understand HTML a lot better than Go, it wouldn’t really matter if I didn’t.
    - skydhash 3mo ago
      > Here’s another example: you’re making a static website. No JavaScript, no interactivity. Linking a huge file consuming clients’s bandwith for no reason. Embedding PII in the html source? And if setting up your own server, misconfiguring it?…
      - Grombobulous 3mo ago
        “Setting up your own server” isn’t part of this, as you’d almost certainly deploy a static site using something simple and serverless. You also don’t need to know how to read HTML to recognize large files. You can catch issues like this with a simple website performance testing tool like pagespeed.web.dev I’m also not sure how PII would enter the HTML source.
    - what 3mo ago
      > I just wanted to move from bash scripts to using cloud SDKs for performance reasons. What is this supposed to mean? How is a “cloud sdk” more performant than a shell script?
      - Grombobulous 3mo ago
        The shell script wasn’t the performance issue, it was the fact that the shell script was calling the AWS CLI rather than the SDK. There’s a bit less waiting around.
  - fzeroracer 3mo ago
    If I'm on call solving a problem another engineer caused and I reach out to them for clarification and they say 'I don't know, the AI wrote it' I am going to advocate for them being fired tomorrow.
- serious_angel 3mo ago
  For example, the following "works": json='{ "left":2, "right":2 }'; result="$( perl -e '($_)=<>; / "left":(\d+), "right":(\d+)/; print $1 + $2, "\n";' <<< "$json"; )"; printf '%s\n' "$result"; Yet, it is literally the same as: printf '%s\n' "$(( 2 + 2 ))";
- xigoi 3mo ago
  Vibe coders seem to have the misconception that if the LLM’s code passes all tests (also generated by the LLM), then it must be correct.
- archargelod 3mo ago
  You can't. You must prove it. And I don't mean that you need a rigorous scientific proof - that would obviously be too hard to do for every single function/library/program. Human developer can work on a program incrementally, ensuring at each step that it is mostly correct. But LLMs can't think, they fake reasoning and explore problem space in random walk until they stumble into something that looks like a solution. And these "solutions" will have hilarious and absolutely unexpected failure modes.
datadrivenangel 3mo ago
"The reality is that code that runs and makes the CI green can still be a bad solution, and engineering has always been about implementing adequate, scalable, and extensible solutions." Adequate often means done and cheap
- solid_fuel 3mo ago
  Disagree, adequate means adequate. Done and cheap is what you call it when a solution is adequate. If the solution isn't adequate, it doesn't matter if it's cheap, because it isn't done.
- DrewADesign 3mo ago
  As long as safe and stable are assumed to be base-level requirements… maybe?
- josephg 3mo ago
  > Adequate often means done and cheap It really, REALLY depends what you're working on. If you're throwing together an internal tool or simple dashboard, it doesn't really matter what the code looks like. But if you're writing software that other programs will depend on, bad design choices ripple out and affect another generation of software. Imagine slop in the linux kernel, in google chrome, or in your compiler or runtime. Its not acceptable. I know a lot of people spend their careers writing end user software and web UIs. AI is increasingly a good choice for this sort of code. But that's not all of us. And its not all of the software being written.
- skydhash 3mo ago
  I was just watching a video about system engineering and the following stucks: Stakeholder needs: What people wants to get done with the product Management needs: How to manage the spending of resources (time, money,…) to create the product Engineering needs: What is the product You have to balance the three. Sometimes it’s simple and easy to get right. Sometimes it’s complex enough, you’re never truly sure until the product is out in the wild. Software is malleable and we can do easily do iterations which is not possible with hardware. But today, we have a skew towards engineering, where the whole focus is to create a solution, whatever that is. No understanding of the problem, no proper allocation of resources, just do something. Even if it is plastering over the crack for the eleventh time.
Aurornis 3mo ago
Even using Fable (while it was briefly available), having it refine a plan, and directing it to make only small incremental changes, I still found reasons to reject its first pass at a lot of work. There was a lot of “You’re right to push back” responses. A lot of incidents where it would creat some giant complex set of abstractions to accomplish something that I could find ways to do much more elegantly and in a more maintainable manner. It’s really eye opening to work with these tools on a codebase you know deeply because these problems are everywhere. However if I opened an unfamiliar project in another language and I wanted to add a little feature with no intention of maintaining it, I’d happily accept the changes and loop until it worked well enough for my temporary needs. The scary middle is when you’re dealing with coworkers who don’t care about anything other than closing tickets and collecting credit. With enough of a token budget you can now wrap loops around an LLM and have it try things until the program appears to work. Ask it to do a code review and then submit the PR without having understood what it was doing. There are a lot of workplaces where there isn’t a good mechanism to push back on this and the tech debt just keeps growing.
- busterarm 3mo ago
  > With enough of a token budget you can now wrap loops around an LLM and have it try things until the program appears to work. Ask it to do a code review and then submit the PR without having understood what it was doing. There are a lot of workplaces where there isn’t a good mechanism to push back on this and the tech debt just keeps growing. I'm not making an argument in favor of people using LLMs for this, but people were doing this before we had LLMs it was just usually a bit slower. I can't even say it usually doesn't work out long term because I worked with a lot of guys who did this and took a ton of Adderall while working practically around the clock. Every incentive structure in the organizations rewarded it along with social credibility from more junior engineers. (The last cowboy I worked with who pulled this shit ended up becoming the most senior engineer in the company, a multi-millionaire and worshipped like a god by 90% of the mostly fresh grads we were hiring). The problem is when invariably these people burn out eventually and leave, they leave a massive vacuum in their stead. Not from load they were carrying but creating. I think the larger the organization I've been at, the more they reward the people making huge commits on nights and weekends. Worse, they could get away with TBRing their shit and merging it without review. LLMs are often all of the bad habits and organizational problems that we already carryied just being speedrun. There are some places doing it right, but they already were.
summerlight 3mo ago
My personal rule of thumb: I am usually okay with agents driving e2e implementations if this won't make life noticeably worse when it does not work. Some analytical code? Perfectly fine. Hobby projects? Fine, though I prefer doing a fun part myself. Refactoring production code generating 10x more revenue than my salary? You'd better be at least understanding what it does.
- resonious 3mo ago
  Yes this is the thing with these new tools. You have to know when to use them and when not to. Good ol' software architecture tricks can also help you slot "vibe coded" components into a larger system safely.
ecshafer 3mo ago
If we rephrased this to "When I reject my coworkers code even if it works" and give the same reasons there would be zero dissent. There is this weird idea that seems to come up with AI that any solution must be good and adequate. Software Engineering is all about rejecting code that works for the right code that works.
- api 3mo ago
  Which means it doesn’t matter if the code is from AI or not. If it’s not good it’s not good.
- mkozlows 3mo ago
  Yeah, but I think there's a difference here: If your coworker puts up code that you don't understand quickly, in most environments people give it an approval, as withholding approval is meant to indicate that there's a problem with the code. It's very rare that you'd actually force them to wait to merge until they've explained the code to your satisfaction. (There are workplaces where that's the norm, I know -- it tends to be a thing with smaller teams with codebases that everyone understands fully, and much less a thing with larger teams where different people have areas of the code they understand more than others.) With AI code, though, it's _your code_ and you can't give it a lgtm, you actually need to dig at it until you do fully understand it, fully agree with it, and could justify it to a hostile reviewer. It's a different level of rigor. Not all engineers apply that rigor, though, which becomes a problem.
  - Agentlien 3mo ago
    On the teams where I've worked which had a proper code review policy not understanding has been explicitly stated as a reason to pause and ask for clarification. You cannot give a pass for code you don't understand.
  - coffeefirst 3mo ago
    What? No… we would jump on a call and talk through it for 5 minutes. I’m not saying you must see into the soul of every line, but “no idea what I’m looking at, LGTM” misses the point of code review. I have never been on a team where that’s okay.
rvz 3mo ago
> Before coding agents, when given a task, I would explore the codebase, think of different solutions, experiment, and only then implement. That could take days of consolidating all that context. When I finally submitted that PR, confidence was higher, and explaining each of my changes to my coworkers was easier. Now we are getting to the point where we are speed-running the deskilling of engineers into comprehension debt and they themselves rapidly losing confidence in reviewing code they did not write. I think this blog post [0] is the best example of what could go entirely wrong and even worse when you do not know the technology. If you cannot explain a change even when "the CI is green" or "all tests passing", I will immediately reject it. Maybe great for vibe coding prototypes, but it all changes when that code is deployed onto mission critical systems. Just ask Amazon with Kiro. [1] [0] https://sketch.dev/blog/our-first-outage-from-llm-written-code https://sketch.dev/blog/our-first-outage-from-llm-written-co... [1] https://www.reuters.com/business/retail-consumer/amazons-cloud-unit-hit-by-least-two-outages-involving-ai-tools-ft-says-2026-02-20/ https://www.reuters.com/business/retail-consumer/amazons-clo...
eranation 3mo ago
LLMs diverge, not converge. They slightly increase entropy if not controlled. While you can have DRY skills and use AI to organize AI (in loops(tm) like Boris does) but eventually if you don’t understand the code, you are taking yourself out of the loop. And not just the job security that’s on the line, it’s the increasing cost for AI to babysit AI. If you or your “loops” (or paperclip, Hermes, gastown, or next in class agents of agents that runs your entire company) let it gradually sneak in slop-debt, the cost to fix it later will become prohibitive. (You can always just rewrite it, but as the race for “feature complete” and “zero backlog” continues, rewriting an ever growing set of new daily table stakes will become an economical moat) TLDR: Keeping your codebase human readable and reason-about-able is not just helping humans to stay relevant. It will save costs for LLMs to maintain it.
- simondotau 3mo ago
  > They slightly increase entropy if not controlled. If you use AI at the very start of a project, replace slightly with greatly. AI loves to write abstractions and indirection and add complexity wherever it can. And it does so really, really, really badly. AI is great at writing procedural code, but it's a world class shit-for-brains at architecture. It has no taste, no restraint, no appreciation of simplicity. And it wouldn't be so bad, except it's ALSO a complete toaster when it comes to naming things.
AmareshHebbar 3mo ago
If I can't explain the code without rereading the diff, I probably shouldn't merge it.
krupan 3mo ago
And again this makes me wonder, is AI really helping if this much review and rework is needed for all the code it writes?
- teaearlgraycold 3mo ago
  Depends on what it’s writing. There are times an LLM saves me a lot of time researching library functionality. Especially with testing frameworks. So many strange and arcane features out there beyond the basics, but not hard to understand what they do once you see the code. On that topic I should say I am careful when reviewing the actual test cases. However if you’re highly familiar with a domain then LLMs are much less useful.
- mkozlows 3mo ago
  Most code they write is obviously fine. Much of the rest isn't obviously fine, but is in fact fine once you've gone through understanding it. But yes, there's some that still benefits from a human eye. (For as long as that's true, "software developer" is still a job. It's not clear for how long it will be true.)
- unknownfuture 3mo ago
  I mean, the reality is a ton of folks in the industry, myself included, are writing glorified CRUD apps in their day jobs. We're building into existing an codebase with established infrastructure and ways of working. What we're building isn't inherently complex or very interesting. Meanwhile, those codebases often require a ton of boilerplate and drudgery to get anything done. In these spaces it's very easy to read and comprehend AI generated output and review it fairly quickly. So the time savings from dealing with all that boilerplate and conforming with all that existing infrastructure are potentially substantial.
cadamsdotcom 3mo ago
If you reject AI code that works then your mindset is still too hands on. Put another way - you still have some loops to work on taking yourself out of. The agent should’ve delivered code that was acceptable as a first pass. Agents respond really well to feedback! They have no ego and they’ll happily improve code if told where and how. But you need to provide the tools that provide that feedback without your involvement - otherwise you can’t scale. All the linting and autoformatting you can put in, is a good start. Next, create custom scripts that check for every single dumb AI-ism you can think of, tell the agent about them, tell it to use them to check its work, and put them in hooks so the harness refuses to let the agent stop until all your linters show no errors. Then, keep iterating basically forever. Any dumb AI-ism you see, make a linter for it, give it to the agent, and enforce it using the harness. I’ve spent months doing this. When I review a PR - which was built by the agent with TDD so it definitely works - I’m no longer asking if it did dumb stuff or confirming it conformed to the architecture or duplicated code or missed opportunities for reuse. That’s all linted for. I don’t worry about duplication or outdated docstrings/comments because the self review caught all that. I mostly read it to look for opportunities to make the feature even better & more useful. If this makes no sense or you disagree it’s possible, my contact details are on my profile and I’ll be happy to give a demo.
- equinumerous 3mo ago
  I am very curious what some of your lint rules look like in practice. In my mind a lot of the AI-isms in my code that I hate are stylistic or a matter of taste, not necessarily something I could write a deterministic rule to check. But I want to hear more. Like, what kind of linters did you create and which were highest impact?
  - cadamsdotcom 3mo ago
    Start at https://github.com/cadamsdotcom/CodeLeash/blob/main/.pre-commit-config.yaml https://github.com/cadamsdotcom/CodeLeash/blob/main/.pre-com... where you’ll see the custom lints. Then have a look at https://github.com/cadamsdotcom/CodeLeash/blob/main/scripts/check_code_quality.py https://github.com/cadamsdotcom/CodeLeash/blob/main/scripts/... (which was test-driven alongside https://github.com/cadamsdotcom/CodeLeash/blob/main/tests/unit/scripts/test_check_code_quality.py https://github.com/cadamsdotcom/CodeLeash/blob/main/tests/un...) The script can exit 2 to block the agent, and whatever it prints to stderr is shown to the agent. That’s a pretty darn flexible way to enforce whatever you like. Despite this being in the codebase I still have no idea what python’s ast stuff is or does - I just let the agent rip, ensured it did TDD and reviewed it all to make sure the tests & code looked reasonable. I didn’t write this code and don’t want to. But I’ve watched it catch hundreds of dumb AI-isms, and watched the agent go “okay” and fix them ;) it’s been paying for itself over and over for months :)
wwind123 3mo ago
I use 3 AI's (Claude, GPT and Gemini) to review each other's design plans and implementation on the same code base. Each often catches problems the others miss. I try to make sure the architecture docs of the code base are refreshed regularly based on recent changes, so it's easier for humans and AI agents to make sense of the code. I also regularly stop all other developments and just focus on auditing the code base with these AI's to make sure they are secure, robust, clean, and well structured and well tested -- some refactoring would be needed most of the time, and it's well worth it. With this approach, nowadays I often merge code from AI without completely understanding what it's doing, but seems the code has been working so far. :)
- BobbyTables2 3mo ago
  You’ve transitioned from “individual contributor” to “manager”! (;->
  - wwind123 3mo ago
    Haha, true! I do sometimes have to steer the discussions between the AI's to the right direction, if they deviate too far away from the real problem, either because they miss some context, or because my original description of the problem was misleading. To do that formally, I have a mechanism built-in the review loop where if a comment on a github issue or PR is signed as "-- Human Reviewer", then all AI agents have to treat the comment as the highest priority item to address.
- jimbobimbo 3mo ago
  This is the way. I use gh copilot and have opus interrogate me and write the plan, then gpt review the plan and provide feedback; repeat this multiple times until gpt is either satisfied or starts to nitpick on unimportant stuff. Then sanity check the plan myself and have gpt implement it. Each implementation is also reviewed by me before merging to master. I complete PRs only when I'm satisfied with the implementation, my feedback is addressed, and I fully understand what is going on. Agents are the replacement for typing and productivity multipliers. I have big picture view of the product, each plan implements only a part of it, scoped to avoid merging unreviwed slop. Probably slower, but result is much better.
julianlam 3mo ago
I think a particular failing with developers embracing AI is fighting the sunk cost fallacy. While you might not have spent as much time putting together a non-working solution, you still did spend time working with the agent to slap together a non-working solution. Being able to step back and say "this was a failure and we need to discard the day's work and start over" is still hard with LLMs.
- mkozlows 3mo ago
  Completely disagree. I think this is one of the big wins of agentic engineering. When you look back at your own completed change and realize that you made it too complicated because your initial abstraction was wrong, you have to debate long and hard about whether it's worth going back and redoing the work -- is the abstraction actually that bad? Would you really get a huge win by changing it, enough to justify spending another day on the task? But with the agent, you know that the change will be relatively quick and easy, so the bar to tell it to shift approaches is much, much lower.
- theshrike79 3mo ago
  If I spent half a day asking an agent to do something and it's a "non-working solution". I can just throw it away. I have sunk close to zero cost in it. I have no emotional attachment to the code. It's like if I 3D-printed something I haven't modelled myself and the print goes wonky. I don't spend days trying to glue and file it back together. I chuck it in the bin and start a new one. But if I had handcrafted the same item over multiple days, of course I'd try to salvage it - because there was a sunk cost of me spending time doing it.
jdw64 3mo ago
Coding with AI eventually comes down to two paths, I've realized. One is using AI exclusively for everything. The other is not using it at all. There is almost no middle ground. The reason is that as the complexity and depth of the problem increase, the code AI generates increasingly follows enterprise level patterns. The deeper the meaning of what I input, the more AI tends to produce code that goes beyond my own area of expertise. For example, a human expert's code is very powerful and deep within their own domain, but when you look at the entire codebase, it's often shallow and uneven outside that domain. But the moment you write code with AI, once you go deep in one part, AI tries to standardize the rest accordingly. This means the entire codebase converges toward enterprise level standard code, which essentially reflects the average patterns of senior programmers who built large scale systems. The problem is this. Human cognitive resources are finite, so we inevitably become shallow outside our own expertise. There is no programmer who can do everything well. And as systems grow in scale, they become more modularized and fragmented, making it impossible to understand the whole system. So what should we do about this? That's always the question. In the end, do I choose not to use AI, finish the project with uneven code outside my domain, and deliver it? Or do I use AI and deliver a program that is uniform and consistent, but not in my own style? I still don't know. I haven't found the answer yet.
- mkozlows 3mo ago
  You can also just use AI and keep the scale of your changes small rather than refactoring the whole app with a change? This isn't super-weird.
  - jdw64 3mo ago
    As you know, the boundary ultimately depends on code quality. The problem is that AI generates code that looks high quality even outside my area of expertise, at least from my perspective. So now the boundary has to be redrawn. Refactoring usually ends up redefining those boundaries. At that point, the question becomes: do I rewrite my own code, or do I reject the AI code? Those are the two choices left. In the end, an exceptionally skilled programmer might be able to keep their core domain intact, but I think the vast majority would find that very difficult. So it might be possible once you cross a certain threshold, but considering the sheer amount of code required to deliver a single modern program, it's hard to know which parts to focus on. However, my perspective might be different because I'm coming from the point of view of delivering a working program, not from the perspective of open source development
OffBeatDev 3mo ago
[dead]
monkamonme 3mo ago
[flagged]
cws_ai_buddy 3mo ago
[flagged]
aimattb 3mo ago
[flagged]
piterrro 3mo ago
I feel the same way, reading AI built feature entire output makes me cognitively overloaded as well - I can only do so many throughout the day. What I found myself doing is operating in two modes: 1. For projects that require my attention, I plan and instruct LLM, when needed will draft some code and ask agent to make it better or finish the mundane part (write code and leave gaps with comments asking agent to finish) 2. Full automode where I use spec driven development and TDD - I only ask for changes based on existing PRD, which agent also have to update. Here I do not look at the code at all. Seems to be working just fine.
panchtatvam 3mo ago
You must accept AI code only if you deem yourself dumber than AI.
- cws_ai_buddy 3mo ago
  [flagged]
philbo 3mo ago
Yesterday I started working on an agent harness that tries to address some of the issues here. What I'm hoping to build ultimately is something that works more like a pair-programming partner than existing harnesses do. I want the user to be an engaged part of the development process all the way through, I don't want the agent disappearing to work on its own. I even want to make it possible for users to swap into the driver role and have the LLM automatically assume the role of navigator when that happens. There's more info in the readme (actually the readme is all that exists so far, I wanted to get the idea straight in my head first): https://gitlab.com/philbooth/opair https://gitlab.com/philbooth/opair Even if nobody else uses it, I hope it will be a useful tool for myself and help me find a way to work with LLMs that doesn't harm my mental models, which is what I feel current harnesses do.
osigurdson 3mo ago
Its hard to find a middle ground between fully understanding everything in a PR vs a vibe coding type approach. Can you understand "just a little bit" of a PR and merge it into a code base you really care about? Is it maybe fine to "mostly understand it" on the other hand? Its definitely a tough call and its impossible to argue that no trade off is being made. LLMs are perfect for quick prototypes, speed runs, learning, etc., but if the code really matters its still not clear cut. I think the definition of what "really matters" is very project dependent of course As an extreme example you would want to understand every line of the code for the control system runs an MRI machine or a jet engine since bugs might mean life or death. Depositing money into the wrong account might not kill anyone but could lead to severe economic losses. But, then again, even problems in far less consequential software may be drastically sub-economic (i.e. saving $1000 on the implementation might cost $10000 if customers aren't happy and fails to re new). Pick your scenario I guess. The problem is, this isn't going to change regardless of how well a new model scores on a benchmark. It seems actually AGI is needed.
moezd 3mo ago
If it's code that you can tolerate being somewhat messy and suboptimal, you can run agents e2e. If it's critical piece of code that has become part of your identity, better do the PR work and scrutinize it well. LLMs are still next token predictors, no matter how much harness, hooks, skills and tools is attached to them. LLMs will only know that these are callable, interpretating the state and mitigation are still best effort.
YongHaoHu 3mo ago
[dead]
whilenot-dev 3mo ago
Titles like these make me always point out the obvious: A working state is the absolute minimum requirement for any code to be merged, isn't it? ...imagine to merge something even though you know that's not working. Besides, this post has nothing specific to code produced by an LLM, and placing AI in the stated reasons feels completely arbitrary, or is rather a fallacy of our times: - I reject [AI] code when I can’t explain the approach in my own words. - I reject [AI] code when the diff is bigger than the problem. - I reject [AI] code when it introduces abstractions before proving they’re needed. - I reject [AI] code when it works locally but makes the system harder to reason about. - I reject [AI] code when I’m trusting the output more than my understanding.
- utopiah 3mo ago
  Fallacy or scapegoat. If management ask for revised KPIs where PRs must be 10x and AI is the "excuse" for this (unrealistic) new demand.
- simondotau 3mo ago
  Well said. Replace [AI] with "junior dev" or "consultancy contractor" and these assertions have always been thus.
- QuiEgo 3mo ago
  I’ve had multiple people say “you don’t work on code anymore, that’s for the AI. You work a level of abstraction above that. As long as you prove it works through testing, the code doesn’t matter anymore. It’s like looking at the assembly the compiler spits out now - who cares?” These are the people who spit out an incredible volume of code with AI, to the point reviews simply can’t keep up. The last person who said this to me works in embedded, where we look at the assembly all the time. Scary.
  - theshrike79 3mo ago
    But if the output matches the duck typing test, does it actually matter what's inside the black box of code? If you're given two embedded devices and both pass the same testing, how would you tell which one was 100% AI code and which was beautifully handcrafted line by line?
danfritz 3mo ago
This resonates a lot with me. I often use AI for the plan and let it propose multiple possible implementations, I often have to point out the glaring easier / logical solution. When implementing its often a lot of misses with a few golden hits. The other day it used flex for a table layout while our app uses tables everywhere sigh. Another typical one is that it tends to prefere frontend aggregation and looping of data instead of letting the database and backend deal with it. Using mix of claude, cursor composer and codex.
- simondotau 3mo ago
  I find Cursor/Composer is really good at mimicking existing code when writing new code. And it will often do so without being asked, but I try to always explicitly mention an existing bit of code for it to read as inspiration (e.g. "use the TPS 2.0 report as a style guide").
edanm 3mo ago
Not that I disagree with anything here, but... I wish it were clearer in these kinds of posts how "I use AI code I don't understand" is so different from "I use libraries written by other people I don't understand", or "I work in a large codebase which was 99% written by other people, and I haven't seen all of it", or even "I use software written by other people I don't understand".
SunboX 3mo ago
I unterstand the reasons, but I don't think so. I have experience in software development over 20 years now and still developing software daily. Nowadays it's nearly 100℅ AI written. It looks good and works. Sure, you have to guide the AI. But this can be done with custom skills, angent files, code quality guards test cases and so on. Maybe the code looks at the end not as I would have written it, maybe something is too complex implemented. But that's true for large developer teams also. At the end it's way faster and it works. I think, everyone who does not adapt to this new workflow is left behind in professional development soon.
- PacificSpecific 3mo ago
  That's cool. Could you share some concrete examples of your successes?
  - SunboX 3mo ago
    One I currently working on (privately) is ecadforge.app The last ones, I worked on in Industry are retail7 apps, Migros Self scanning client, EDEKA, LIDL and so on customer facing apps. My private interest is more in electronics.
    - PacificSpecific 3mo ago
      Ah interesting. We are in very different domains haha. I had to search a lot of terms. Thanks for sharing!
  - SunboX 3mo ago
    Otherwise I quickly did some smaller projects lately, mainly AI driven. Just take a look at my list here: https://github.com/SunboX?tab=repositories https://github.com/SunboX?tab=repositories
- tom2026hn 3mo ago
  Do you still need to review the code? LLMs generate code too quickly for humans to keep up.
tarkin2 3mo ago
Code that kills your project is code that works, and is either misunderstood or unmaintainable And the industry is rushing towards it, whilst failing to train people who are able to fix it
CraigJPerry 3mo ago
The bottleneck when using a "faster keyboard" is understanding. We have a tool for this in compsci. Not having to fully understand something in order to successfully exploit it is a staple of computer science; we use abstractions to help us reason at a higher level. You don't necessarily always have to understand the nuance involved in selecting a hash function just to put and get some items in a hash map. Specifically, when are these cases where you don't need to go that deep? Are there similar scenarios for ai written code? I'm more interested right now in what does that abstraction look like for AI generated code. Is there some reasonable solution wherein a sandboxed component in the enterprise architecture has various attributes (e.g. the bytes i stuff into this file store component are always the exact bytes i get back from it) confirmed by methods other than a human reading its code? Those methods, are they cheaper, faster, safer than just having a human do it? If your enterprise architects have to read every line of code in your system today then i'd claim your architecture practices have room to mature. What can derived from that, and in which scenarios, for the purposes of safely leveraging immutable write-only code? I'm not interested in evolving the code (lines of code spent to solve a business problem was never an asset, it was always a cost) if it wasn't hand crafted by a human, i still have the requirements so i can just regenerate the entire thing with the revised requirement.
- theshrike79 3mo ago
  Visualisation tooling IMO is the way to reason with this. You don't look at the code, but use tooling to create a chart of the calls, data models etc. Then you can look at that and see the complexity. ...and we already had these tools in the early 2000s, when (can't remember which) no-code fad was running about. You know the ones where you just draw the boxes and lines and poof the code is generated =) There were also tools that did the reverse.
neonstatic 3mo ago
The more I look into it, the more I am convinced, that I don't want any AI generated code in my project. I find the LLM useful to talk things through. It can offer some interesting takes during the design and acts as a decent rubber duck companion when debugging. It is also very helpful when I need some help with syntax and/or feature discovery. > I reject AI code when I can’t explain the approach in my own words. I think that's the key problem. LLMs turn code into big, black boxes. Sure, theoretically nothing stops me from reading all that code. I don't, however, because it's wasted effort. The time it takes me to really understand the code is IMO better spent just writing it myself. Once written, I have a very good understanding. Read ten times, not so much. It reminds me of pen and paper. Journaling the old way remains the best way to learn something, but writing on a computer is much more convenient.
sltr 3mo ago
It's kind of like the saying "the right note played at the wrong time is wrong." An implementation that works today but could break in the future is wrong. I'm not willing to treat an LLM as an oracle that knows the difference. It certainly hasn't earned that trust.
terekhindc 3mo ago
[flagged]
mdavid626 3mo ago
The problem is if your coworkers don’t do it this way.
- vnbrs 3mo ago
  That’s why I think code reviews should be human first. That’s where we get the most value out of it. AI code reviews are still good though, but they should be treated as linters in your CI pipeline.
  - mdavid626 3mo ago
    I haven’t seen any useful AI code review so far.
swordsith 3mo ago
why I reject thing that makes other peoples lives easier even if it makes peoples lives easier.
- vnbrs 3mo ago
  That is not what I said in that post.
Avrio15272 3mo ago
[dead]
jameslaneyno9 3mo ago
The sycophancy problem disappears when you stop pushing back on the agent and start having a human review the plan before the agent writes anything. Push back on a human who wrote the plan and their agreement or disagreement will actually be meaningfull. Push back on the agent and you get "you're absolutely right..". The plan also solves "I can't explain this code" because you wrote the plan before the build, so you can explain it. After tracking some internal metrics recently we found plan review costs 0.7 hours on average compared to PR review that costs 16 hours. We rejected 13 out of 165 plans meaning no code was written. The one gap this doesn't close is that the agent drifts from the plan. We run a separate adversarial check that compares the diff against the approved plan and flags anything the plan didn't specify. That catches scope drift without reading every line.
icoderbilal 3mo ago
[flagged]

237 points by vnbrs 3mo ago

codelong888 3mo ago

[flagged]

deleted 3mo ago

[deleted]

_wire_ 3mo ago

"Even if it works?" How do you verify that it works?

p1024k 3mo ago
According to the author's intention, it is the code that he cannot understand or control. Even if the solution provided by the AI works, he will not adopt it. This is unless he can understand or control it. This should be an assumption. However, if AI provides a solution, as the person using AI, one should conduct research before making a decision. This is not in conflict with or hindered by the use of the ideas provided by AI.
- andyfilms1 3mo ago
  I will say--as someone who has fielded late night troubleshooting calls--I totally understand OP's point of view. It's reasonable to expect that you will be able to answer questions about something that you ship, or brainstorm ways to solve a problem a customer is encountering while using something you provided them. The obvious counterargument is "well, just ask the AI for those answers," but the AI lacks the context and experience that you have. Sometimes, genuinely, the user really is just "holding it wrong," but none of the current AI models would ever admit that, and you'd spend hours trying to solve an unsolvable problem.
- Grombobulous 3mo ago
  I think this policy is probably more prescriptive than I would go with myself. I like to think of my risk tolerance first to help make that determination. For example, I use a vibecoded internal tool written in Go. I don’t even know how to write Go. Haven’t read a single line of the code. I just wanted to move from bash scripts to using cloud SDKs for performance reasons. But the internal tool is a convenience tool, and you can do everything it does using alternative methods. So if it break, there is no real negative impact besides personal convenience of anyone using it. There’s some documentation on how to do everything manually if needed. Here’s another example: you’re making a static website. No JavaScript, no interactivity. Truly, what could go wrong? And while I do understand HTML a lot better than Go, it wouldn’t really matter if I didn’t.
  - skydhash 3mo ago
    > Here’s another example: you’re making a static website. No JavaScript, no interactivity. Linking a huge file consuming clients’s bandwith for no reason. Embedding PII in the html source? And if setting up your own server, misconfiguring it?…
    - Grombobulous 3mo ago
      “Setting up your own server” isn’t part of this, as you’d almost certainly deploy a static site using something simple and serverless. You also don’t need to know how to read HTML to recognize large files. You can catch issues like this with a simple website performance testing tool like pagespeed.web.dev I’m also not sure how PII would enter the HTML source.
  - what 3mo ago
    > I just wanted to move from bash scripts to using cloud SDKs for performance reasons. What is this supposed to mean? How is a “cloud sdk” more performant than a shell script?
    - Grombobulous 3mo ago
      The shell script wasn’t the performance issue, it was the fact that the shell script was calling the AWS CLI rather than the SDK. There’s a bit less waiting around.
- fzeroracer 3mo ago
  If I'm on call solving a problem another engineer caused and I reach out to them for clarification and they say 'I don't know, the AI wrote it' I am going to advocate for them being fired tomorrow.
serious_angel 3mo ago
For example, the following "works": json='{ "left":2, "right":2 }'; result="$( perl -e '($_)=<>; / "left":(\d+), "right":(\d+)/; print $1 + $2, "\n";' <<< "$json"; )"; printf '%s\n' "$result"; Yet, it is literally the same as: printf '%s\n' "$(( 2 + 2 ))";
xigoi 3mo ago
Vibe coders seem to have the misconception that if the LLM’s code passes all tests (also generated by the LLM), then it must be correct.
archargelod 3mo ago
You can't. You must prove it. And I don't mean that you need a rigorous scientific proof - that would obviously be too hard to do for every single function/library/program. Human developer can work on a program incrementally, ensuring at each step that it is mostly correct. But LLMs can't think, they fake reasoning and explore problem space in random walk until they stumble into something that looks like a solution. And these "solutions" will have hilarious and absolutely unexpected failure modes.

datadrivenangel 3mo ago

"The reality is that code that runs and makes the CI green can still be a bad solution, and engineering has always been about implementing adequate, scalable, and extensible solutions." Adequate often means done and cheap

solid_fuel 3mo ago
Disagree, adequate means adequate. Done and cheap is what you call it when a solution is adequate. If the solution isn't adequate, it doesn't matter if it's cheap, because it isn't done.
DrewADesign 3mo ago
As long as safe and stable are assumed to be base-level requirements… maybe?
josephg 3mo ago
> Adequate often means done and cheap It really, REALLY depends what you're working on. If you're throwing together an internal tool or simple dashboard, it doesn't really matter what the code looks like. But if you're writing software that other programs will depend on, bad design choices ripple out and affect another generation of software. Imagine slop in the linux kernel, in google chrome, or in your compiler or runtime. Its not acceptable. I know a lot of people spend their careers writing end user software and web UIs. AI is increasingly a good choice for this sort of code. But that's not all of us. And its not all of the software being written.
skydhash 3mo ago
I was just watching a video about system engineering and the following stucks: Stakeholder needs: What people wants to get done with the product Management needs: How to manage the spending of resources (time, money,…) to create the product Engineering needs: What is the product You have to balance the three. Sometimes it’s simple and easy to get right. Sometimes it’s complex enough, you’re never truly sure until the product is out in the wild. Software is malleable and we can do easily do iterations which is not possible with hardware. But today, we have a skew towards engineering, where the whole focus is to create a solution, whatever that is. No understanding of the problem, no proper allocation of resources, just do something. Even if it is plastering over the crack for the eleventh time.

Aurornis 3mo ago

Even using Fable (while it was briefly available), having it refine a plan, and directing it to make only small incremental changes, I still found reasons to reject its first pass at a lot of work. There was a lot of “You’re right to push back” responses. A lot of incidents where it would creat some giant complex set of abstractions to accomplish something that I could find ways to do much more elegantly and in a more maintainable manner. It’s really eye opening to work with these tools on a codebase you know deeply because these problems are everywhere. However if I opened an unfamiliar project in another language and I wanted to add a little feature with no intention of maintaining it, I’d happily accept the changes and loop until it worked well enough for my temporary needs. The scary middle is when you’re dealing with coworkers who don’t care about anything other than closing tickets and collecting credit. With enough of a token budget you can now wrap loops around an LLM and have it try things until the program appears to work. Ask it to do a code review and then submit the PR without having understood what it was doing. There are a lot of workplaces where there isn’t a good mechanism to push back on this and the tech debt just keeps growing.

busterarm 3mo ago
> With enough of a token budget you can now wrap loops around an LLM and have it try things until the program appears to work. Ask it to do a code review and then submit the PR without having understood what it was doing. There are a lot of workplaces where there isn’t a good mechanism to push back on this and the tech debt just keeps growing. I'm not making an argument in favor of people using LLMs for this, but people were doing this before we had LLMs it was just usually a bit slower. I can't even say it usually doesn't work out long term because I worked with a lot of guys who did this and took a ton of Adderall while working practically around the clock. Every incentive structure in the organizations rewarded it along with social credibility from more junior engineers. (The last cowboy I worked with who pulled this shit ended up becoming the most senior engineer in the company, a multi-millionaire and worshipped like a god by 90% of the mostly fresh grads we were hiring). The problem is when invariably these people burn out eventually and leave, they leave a massive vacuum in their stead. Not from load they were carrying but creating. I think the larger the organization I've been at, the more they reward the people making huge commits on nights and weekends. Worse, they could get away with TBRing their shit and merging it without review. LLMs are often all of the bad habits and organizational problems that we already carryied just being speedrun. There are some places doing it right, but they already were.

summerlight 3mo ago

My personal rule of thumb: I am usually okay with agents driving e2e implementations if this won't make life noticeably worse when it does not work. Some analytical code? Perfectly fine. Hobby projects? Fine, though I prefer doing a fun part myself. Refactoring production code generating 10x more revenue than my salary? You'd better be at least understanding what it does.

resonious 3mo ago
Yes this is the thing with these new tools. You have to know when to use them and when not to. Good ol' software architecture tricks can also help you slot "vibe coded" components into a larger system safely.

ecshafer 3mo ago

If we rephrased this to "When I reject my coworkers code even if it works" and give the same reasons there would be zero dissent. There is this weird idea that seems to come up with AI that any solution must be good and adequate. Software Engineering is all about rejecting code that works for the right code that works.

api 3mo ago
Which means it doesn’t matter if the code is from AI or not. If it’s not good it’s not good.
mkozlows 3mo ago
Yeah, but I think there's a difference here: If your coworker puts up code that you don't understand quickly, in most environments people give it an approval, as withholding approval is meant to indicate that there's a problem with the code. It's very rare that you'd actually force them to wait to merge until they've explained the code to your satisfaction. (There are workplaces where that's the norm, I know -- it tends to be a thing with smaller teams with codebases that everyone understands fully, and much less a thing with larger teams where different people have areas of the code they understand more than others.) With AI code, though, it's _your code_ and you can't give it a lgtm, you actually need to dig at it until you do fully understand it, fully agree with it, and could justify it to a hostile reviewer. It's a different level of rigor. Not all engineers apply that rigor, though, which becomes a problem.
- Agentlien 3mo ago
  On the teams where I've worked which had a proper code review policy not understanding has been explicitly stated as a reason to pause and ask for clarification. You cannot give a pass for code you don't understand.
- coffeefirst 3mo ago
  What? No… we would jump on a call and talk through it for 5 minutes. I’m not saying you must see into the soul of every line, but “no idea what I’m looking at, LGTM” misses the point of code review. I have never been on a team where that’s okay.

rvz 3mo ago

> Before coding agents, when given a task, I would explore the codebase, think of different solutions, experiment, and only then implement. That could take days of consolidating all that context. When I finally submitted that PR, confidence was higher, and explaining each of my changes to my coworkers was easier. Now we are getting to the point where we are speed-running the deskilling of engineers into comprehension debt and they themselves rapidly losing confidence in reviewing code they did not write. I think this blog post [0] is the best example of what could go entirely wrong and even worse when you do not know the technology. If you cannot explain a change even when "the CI is green" or "all tests passing", I will immediately reject it. Maybe great for vibe coding prototypes, but it all changes when that code is deployed onto mission critical systems. Just ask Amazon with Kiro. [1] [0] https://sketch.dev/blog/our-first-outage-from-llm-written-code https://sketch.dev/blog/our-first-outage-from-llm-written-co... [1] https://www.reuters.com/business/retail-consumer/amazons-cloud-unit-hit-by-least-two-outages-involving-ai-tools-ft-says-2026-02-20/ https://www.reuters.com/business/retail-consumer/amazons-clo...

eranation 3mo ago

LLMs diverge, not converge. They slightly increase entropy if not controlled. While you can have DRY skills and use AI to organize AI (in loops(tm) like Boris does) but eventually if you don’t understand the code, you are taking yourself out of the loop. And not just the job security that’s on the line, it’s the increasing cost for AI to babysit AI. If you or your “loops” (or paperclip, Hermes, gastown, or next in class agents of agents that runs your entire company) let it gradually sneak in slop-debt, the cost to fix it later will become prohibitive. (You can always just rewrite it, but as the race for “feature complete” and “zero backlog” continues, rewriting an ever growing set of new daily table stakes will become an economical moat) TLDR: Keeping your codebase human readable and reason-about-able is not just helping humans to stay relevant. It will save costs for LLMs to maintain it.

simondotau 3mo ago
> They slightly increase entropy if not controlled. If you use AI at the very start of a project, replace slightly with greatly. AI loves to write abstractions and indirection and add complexity wherever it can. And it does so really, really, really badly. AI is great at writing procedural code, but it's a world class shit-for-brains at architecture. It has no taste, no restraint, no appreciation of simplicity. And it wouldn't be so bad, except it's ALSO a complete toaster when it comes to naming things.

AmareshHebbar 3mo ago

If I can't explain the code without rereading the diff, I probably shouldn't merge it.

krupan 3mo ago

And again this makes me wonder, is AI really helping if this much review and rework is needed for all the code it writes?

teaearlgraycold 3mo ago
Depends on what it’s writing. There are times an LLM saves me a lot of time researching library functionality. Especially with testing frameworks. So many strange and arcane features out there beyond the basics, but not hard to understand what they do once you see the code. On that topic I should say I am careful when reviewing the actual test cases. However if you’re highly familiar with a domain then LLMs are much less useful.
mkozlows 3mo ago
Most code they write is obviously fine. Much of the rest isn't obviously fine, but is in fact fine once you've gone through understanding it. But yes, there's some that still benefits from a human eye. (For as long as that's true, "software developer" is still a job. It's not clear for how long it will be true.)
unknownfuture 3mo ago
I mean, the reality is a ton of folks in the industry, myself included, are writing glorified CRUD apps in their day jobs. We're building into existing an codebase with established infrastructure and ways of working. What we're building isn't inherently complex or very interesting. Meanwhile, those codebases often require a ton of boilerplate and drudgery to get anything done. In these spaces it's very easy to read and comprehend AI generated output and review it fairly quickly. So the time savings from dealing with all that boilerplate and conforming with all that existing infrastructure are potentially substantial.

cadamsdotcom 3mo ago

If you reject AI code that works then your mindset is still too hands on. Put another way - you still have some loops to work on taking yourself out of. The agent should’ve delivered code that was acceptable as a first pass. Agents respond really well to feedback! They have no ego and they’ll happily improve code if told where and how. But you need to provide the tools that provide that feedback without your involvement - otherwise you can’t scale. All the linting and autoformatting you can put in, is a good start. Next, create custom scripts that check for every single dumb AI-ism you can think of, tell the agent about them, tell it to use them to check its work, and put them in hooks so the harness refuses to let the agent stop until all your linters show no errors. Then, keep iterating basically forever. Any dumb AI-ism you see, make a linter for it, give it to the agent, and enforce it using the harness. I’ve spent months doing this. When I review a PR - which was built by the agent with TDD so it definitely works - I’m no longer asking if it did dumb stuff or confirming it conformed to the architecture or duplicated code or missed opportunities for reuse. That’s all linted for. I don’t worry about duplication or outdated docstrings/comments because the self review caught all that. I mostly read it to look for opportunities to make the feature even better & more useful. If this makes no sense or you disagree it’s possible, my contact details are on my profile and I’ll be happy to give a demo.

equinumerous 3mo ago
I am very curious what some of your lint rules look like in practice. In my mind a lot of the AI-isms in my code that I hate are stylistic or a matter of taste, not necessarily something I could write a deterministic rule to check. But I want to hear more. Like, what kind of linters did you create and which were highest impact?
- cadamsdotcom 3mo ago
  Start at https://github.com/cadamsdotcom/CodeLeash/blob/main/.pre-commit-config.yaml https://github.com/cadamsdotcom/CodeLeash/blob/main/.pre-com... where you’ll see the custom lints. Then have a look at https://github.com/cadamsdotcom/CodeLeash/blob/main/scripts/check_code_quality.py https://github.com/cadamsdotcom/CodeLeash/blob/main/scripts/... (which was test-driven alongside https://github.com/cadamsdotcom/CodeLeash/blob/main/tests/unit/scripts/test_check_code_quality.py https://github.com/cadamsdotcom/CodeLeash/blob/main/tests/un...) The script can exit 2 to block the agent, and whatever it prints to stderr is shown to the agent. That’s a pretty darn flexible way to enforce whatever you like. Despite this being in the codebase I still have no idea what python’s ast stuff is or does - I just let the agent rip, ensured it did TDD and reviewed it all to make sure the tests & code looked reasonable. I didn’t write this code and don’t want to. But I’ve watched it catch hundreds of dumb AI-isms, and watched the agent go “okay” and fix them ;) it’s been paying for itself over and over for months :)

wwind123 3mo ago

I use 3 AI's (Claude, GPT and Gemini) to review each other's design plans and implementation on the same code base. Each often catches problems the others miss. I try to make sure the architecture docs of the code base are refreshed regularly based on recent changes, so it's easier for humans and AI agents to make sense of the code. I also regularly stop all other developments and just focus on auditing the code base with these AI's to make sure they are secure, robust, clean, and well structured and well tested -- some refactoring would be needed most of the time, and it's well worth it. With this approach, nowadays I often merge code from AI without completely understanding what it's doing, but seems the code has been working so far. :)

BobbyTables2 3mo ago
You’ve transitioned from “individual contributor” to “manager”! (;->
- wwind123 3mo ago
  Haha, true! I do sometimes have to steer the discussions between the AI's to the right direction, if they deviate too far away from the real problem, either because they miss some context, or because my original description of the problem was misleading. To do that formally, I have a mechanism built-in the review loop where if a comment on a github issue or PR is signed as "-- Human Reviewer", then all AI agents have to treat the comment as the highest priority item to address.
jimbobimbo 3mo ago
This is the way. I use gh copilot and have opus interrogate me and write the plan, then gpt review the plan and provide feedback; repeat this multiple times until gpt is either satisfied or starts to nitpick on unimportant stuff. Then sanity check the plan myself and have gpt implement it. Each implementation is also reviewed by me before merging to master. I complete PRs only when I'm satisfied with the implementation, my feedback is addressed, and I fully understand what is going on. Agents are the replacement for typing and productivity multipliers. I have big picture view of the product, each plan implements only a part of it, scoped to avoid merging unreviwed slop. Probably slower, but result is much better.

julianlam 3mo ago

I think a particular failing with developers embracing AI is fighting the sunk cost fallacy. While you might not have spent as much time putting together a non-working solution, you still did spend time working with the agent to slap together a non-working solution. Being able to step back and say "this was a failure and we need to discard the day's work and start over" is still hard with LLMs.

mkozlows 3mo ago
Completely disagree. I think this is one of the big wins of agentic engineering. When you look back at your own completed change and realize that you made it too complicated because your initial abstraction was wrong, you have to debate long and hard about whether it's worth going back and redoing the work -- is the abstraction actually that bad? Would you really get a huge win by changing it, enough to justify spending another day on the task? But with the agent, you know that the change will be relatively quick and easy, so the bar to tell it to shift approaches is much, much lower.
theshrike79 3mo ago
If I spent half a day asking an agent to do something and it's a "non-working solution". I can just throw it away. I have sunk close to zero cost in it. I have no emotional attachment to the code. It's like if I 3D-printed something I haven't modelled myself and the print goes wonky. I don't spend days trying to glue and file it back together. I chuck it in the bin and start a new one. But if I had handcrafted the same item over multiple days, of course I'd try to salvage it - because there was a sunk cost of me spending time doing it.

jdw64 3mo ago

Coding with AI eventually comes down to two paths, I've realized. One is using AI exclusively for everything. The other is not using it at all. There is almost no middle ground. The reason is that as the complexity and depth of the problem increase, the code AI generates increasingly follows enterprise level patterns. The deeper the meaning of what I input, the more AI tends to produce code that goes beyond my own area of expertise. For example, a human expert's code is very powerful and deep within their own domain, but when you look at the entire codebase, it's often shallow and uneven outside that domain. But the moment you write code with AI, once you go deep in one part, AI tries to standardize the rest accordingly. This means the entire codebase converges toward enterprise level standard code, which essentially reflects the average patterns of senior programmers who built large scale systems. The problem is this. Human cognitive resources are finite, so we inevitably become shallow outside our own expertise. There is no programmer who can do everything well. And as systems grow in scale, they become more modularized and fragmented, making it impossible to understand the whole system. So what should we do about this? That's always the question. In the end, do I choose not to use AI, finish the project with uneven code outside my domain, and deliver it? Or do I use AI and deliver a program that is uniform and consistent, but not in my own style? I still don't know. I haven't found the answer yet.

mkozlows 3mo ago
You can also just use AI and keep the scale of your changes small rather than refactoring the whole app with a change? This isn't super-weird.
- jdw64 3mo ago
  As you know, the boundary ultimately depends on code quality. The problem is that AI generates code that looks high quality even outside my area of expertise, at least from my perspective. So now the boundary has to be redrawn. Refactoring usually ends up redefining those boundaries. At that point, the question becomes: do I rewrite my own code, or do I reject the AI code? Those are the two choices left. In the end, an exceptionally skilled programmer might be able to keep their core domain intact, but I think the vast majority would find that very difficult. So it might be possible once you cross a certain threshold, but considering the sheer amount of code required to deliver a single modern program, it's hard to know which parts to focus on. However, my perspective might be different because I'm coming from the point of view of delivering a working program, not from the perspective of open source development

OffBeatDev 3mo ago

[dead]

monkamonme 3mo ago

[flagged]

cws_ai_buddy 3mo ago

[flagged]

aimattb 3mo ago

[flagged]

piterrro 3mo ago

I feel the same way, reading AI built feature entire output makes me cognitively overloaded as well - I can only do so many throughout the day. What I found myself doing is operating in two modes: 1. For projects that require my attention, I plan and instruct LLM, when needed will draft some code and ask agent to make it better or finish the mundane part (write code and leave gaps with comments asking agent to finish) 2. Full automode where I use spec driven development and TDD - I only ask for changes based on existing PRD, which agent also have to update. Here I do not look at the code at all. Seems to be working just fine.

panchtatvam 3mo ago

You must accept AI code only if you deem yourself dumber than AI.

cws_ai_buddy 3mo ago
[flagged]

philbo 3mo ago

Yesterday I started working on an agent harness that tries to address some of the issues here. What I'm hoping to build ultimately is something that works more like a pair-programming partner than existing harnesses do. I want the user to be an engaged part of the development process all the way through, I don't want the agent disappearing to work on its own. I even want to make it possible for users to swap into the driver role and have the LLM automatically assume the role of navigator when that happens. There's more info in the readme (actually the readme is all that exists so far, I wanted to get the idea straight in my head first): https://gitlab.com/philbooth/opair https://gitlab.com/philbooth/opair Even if nobody else uses it, I hope it will be a useful tool for myself and help me find a way to work with LLMs that doesn't harm my mental models, which is what I feel current harnesses do.

osigurdson 3mo ago

Its hard to find a middle ground between fully understanding everything in a PR vs a vibe coding type approach. Can you understand "just a little bit" of a PR and merge it into a code base you really care about? Is it maybe fine to "mostly understand it" on the other hand? Its definitely a tough call and its impossible to argue that no trade off is being made. LLMs are perfect for quick prototypes, speed runs, learning, etc., but if the code really matters its still not clear cut. I think the definition of what "really matters" is very project dependent of course As an extreme example you would want to understand every line of the code for the control system runs an MRI machine or a jet engine since bugs might mean life or death. Depositing money into the wrong account might not kill anyone but could lead to severe economic losses. But, then again, even problems in far less consequential software may be drastically sub-economic (i.e. saving $1000 on the implementation might cost $10000 if customers aren't happy and fails to re new). Pick your scenario I guess. The problem is, this isn't going to change regardless of how well a new model scores on a benchmark. It seems actually AGI is needed.

moezd 3mo ago

If it's code that you can tolerate being somewhat messy and suboptimal, you can run agents e2e. If it's critical piece of code that has become part of your identity, better do the PR work and scrutinize it well. LLMs are still next token predictors, no matter how much harness, hooks, skills and tools is attached to them. LLMs will only know that these are callable, interpretating the state and mitigation are still best effort.

YongHaoHu 3mo ago

[dead]

whilenot-dev 3mo ago

Titles like these make me always point out the obvious: A working state is the absolute minimum requirement for any code to be merged, isn't it? ...imagine to merge something even though you know that's not working. Besides, this post has nothing specific to code produced by an LLM, and placing AI in the stated reasons feels completely arbitrary, or is rather a fallacy of our times: - I reject [AI] code when I can’t explain the approach in my own words. - I reject [AI] code when the diff is bigger than the problem. - I reject [AI] code when it introduces abstractions before proving they’re needed. - I reject [AI] code when it works locally but makes the system harder to reason about. - I reject [AI] code when I’m trusting the output more than my understanding.

utopiah 3mo ago
Fallacy or scapegoat. If management ask for revised KPIs where PRs must be 10x and AI is the "excuse" for this (unrealistic) new demand.
simondotau 3mo ago
Well said. Replace [AI] with "junior dev" or "consultancy contractor" and these assertions have always been thus.
QuiEgo 3mo ago
I’ve had multiple people say “you don’t work on code anymore, that’s for the AI. You work a level of abstraction above that. As long as you prove it works through testing, the code doesn’t matter anymore. It’s like looking at the assembly the compiler spits out now - who cares?” These are the people who spit out an incredible volume of code with AI, to the point reviews simply can’t keep up. The last person who said this to me works in embedded, where we look at the assembly all the time. Scary.
- theshrike79 3mo ago
  But if the output matches the duck typing test, does it actually matter what's inside the black box of code? If you're given two embedded devices and both pass the same testing, how would you tell which one was 100% AI code and which was beautifully handcrafted line by line?

danfritz 3mo ago

This resonates a lot with me. I often use AI for the plan and let it propose multiple possible implementations, I often have to point out the glaring easier / logical solution. When implementing its often a lot of misses with a few golden hits. The other day it used flex for a table layout while our app uses tables everywhere sigh. Another typical one is that it tends to prefere frontend aggregation and looping of data instead of letting the database and backend deal with it. Using mix of claude, cursor composer and codex.

simondotau 3mo ago
I find Cursor/Composer is really good at mimicking existing code when writing new code. And it will often do so without being asked, but I try to always explicitly mention an existing bit of code for it to read as inspiration (e.g. "use the TPS 2.0 report as a style guide").

edanm 3mo ago

Not that I disagree with anything here, but... I wish it were clearer in these kinds of posts how "I use AI code I don't understand" is so different from "I use libraries written by other people I don't understand", or "I work in a large codebase which was 99% written by other people, and I haven't seen all of it", or even "I use software written by other people I don't understand".

SunboX 3mo ago

I unterstand the reasons, but I don't think so. I have experience in software development over 20 years now and still developing software daily. Nowadays it's nearly 100℅ AI written. It looks good and works. Sure, you have to guide the AI. But this can be done with custom skills, angent files, code quality guards test cases and so on. Maybe the code looks at the end not as I would have written it, maybe something is too complex implemented. But that's true for large developer teams also. At the end it's way faster and it works. I think, everyone who does not adapt to this new workflow is left behind in professional development soon.

PacificSpecific 3mo ago
That's cool. Could you share some concrete examples of your successes?
- SunboX 3mo ago
  One I currently working on (privately) is ecadforge.app The last ones, I worked on in Industry are retail7 apps, Migros Self scanning client, EDEKA, LIDL and so on customer facing apps. My private interest is more in electronics.
  - PacificSpecific 3mo ago
    Ah interesting. We are in very different domains haha. I had to search a lot of terms. Thanks for sharing!
- SunboX 3mo ago
  Otherwise I quickly did some smaller projects lately, mainly AI driven. Just take a look at my list here: https://github.com/SunboX?tab=repositories https://github.com/SunboX?tab=repositories
tom2026hn 3mo ago
Do you still need to review the code? LLMs generate code too quickly for humans to keep up.

tarkin2 3mo ago

Code that kills your project is code that works, and is either misunderstood or unmaintainable And the industry is rushing towards it, whilst failing to train people who are able to fix it

CraigJPerry 3mo ago

The bottleneck when using a "faster keyboard" is understanding. We have a tool for this in compsci. Not having to fully understand something in order to successfully exploit it is a staple of computer science; we use abstractions to help us reason at a higher level. You don't necessarily always have to understand the nuance involved in selecting a hash function just to put and get some items in a hash map. Specifically, when are these cases where you don't need to go that deep? Are there similar scenarios for ai written code? I'm more interested right now in what does that abstraction look like for AI generated code. Is there some reasonable solution wherein a sandboxed component in the enterprise architecture has various attributes (e.g. the bytes i stuff into this file store component are always the exact bytes i get back from it) confirmed by methods other than a human reading its code? Those methods, are they cheaper, faster, safer than just having a human do it? If your enterprise architects have to read every line of code in your system today then i'd claim your architecture practices have room to mature. What can derived from that, and in which scenarios, for the purposes of safely leveraging immutable write-only code? I'm not interested in evolving the code (lines of code spent to solve a business problem was never an asset, it was always a cost) if it wasn't hand crafted by a human, i still have the requirements so i can just regenerate the entire thing with the revised requirement.

theshrike79 3mo ago
Visualisation tooling IMO is the way to reason with this. You don't look at the code, but use tooling to create a chart of the calls, data models etc. Then you can look at that and see the complexity. ...and we already had these tools in the early 2000s, when (can't remember which) no-code fad was running about. You know the ones where you just draw the boxes and lines and poof the code is generated =) There were also tools that did the reverse.

neonstatic 3mo ago

The more I look into it, the more I am convinced, that I don't want any AI generated code in my project. I find the LLM useful to talk things through. It can offer some interesting takes during the design and acts as a decent rubber duck companion when debugging. It is also very helpful when I need some help with syntax and/or feature discovery. > I reject AI code when I can’t explain the approach in my own words. I think that's the key problem. LLMs turn code into big, black boxes. Sure, theoretically nothing stops me from reading all that code. I don't, however, because it's wasted effort. The time it takes me to really understand the code is IMO better spent just writing it myself. Once written, I have a very good understanding. Read ten times, not so much. It reminds me of pen and paper. Journaling the old way remains the best way to learn something, but writing on a computer is much more convenient.

sltr 3mo ago

It's kind of like the saying "the right note played at the wrong time is wrong." An implementation that works today but could break in the future is wrong. I'm not willing to treat an LLM as an oracle that knows the difference. It certainly hasn't earned that trust.

terekhindc 3mo ago

[flagged]

mdavid626 3mo ago

The problem is if your coworkers don’t do it this way.

vnbrs 3mo ago
That’s why I think code reviews should be human first. That’s where we get the most value out of it. AI code reviews are still good though, but they should be treated as linters in your CI pipeline.
- mdavid626 3mo ago
  I haven’t seen any useful AI code review so far.

swordsith 3mo ago

why I reject thing that makes other peoples lives easier even if it makes peoples lives easier.

vnbrs 3mo ago
That is not what I said in that post.

Avrio15272 3mo ago

[dead]

jameslaneyno9 3mo ago

The sycophancy problem disappears when you stop pushing back on the agent and start having a human review the plan before the agent writes anything. Push back on a human who wrote the plan and their agreement or disagreement will actually be meaningfull. Push back on the agent and you get "you're absolutely right..". The plan also solves "I can't explain this code" because you wrote the plan before the build, so you can explain it. After tracking some internal metrics recently we found plan review costs 0.7 hours on average compared to PR review that costs 16 hours. We rejected 13 out of 165 plans meaning no code was written. The one gap this doesn't close is that the agent drifts from the plan. We run a separate adversarial check that compares the diff against the approved plan and flags anything the plan didn't specify. That catches scope drift without reading every line.

icoderbilal 3mo ago

[flagged]