4 ms·

This is real. I’ve seen some baffling bugs in prompt based stop hook behavior. When I investigated I found the docs and implementation are completely out of sy

by eunoia 8mo ago

This is real. I’ve seen some baffling bugs in prompt based stop hook behavior. When I investigated I found the docs and implementation are completely out of sync, but the implementation doesn’t work anyway. Then I went poking on GitHub and found a vibed fix diff that changed the behavior in a totally new direction (it did not update the documentation). Seems like everyone over there is vibing and no one is rationalizing the whole.

skerit 8mo ago
I switched to OpenCode, away from Claude-Code, because Claude-Code is _so_ buggy.
heliumtera 8mo ago
>Seems like everyone over there is vibing and no one is rationalizing the whole. Claude Code creator literally brags about running 10 agents in parallel 24/7. It doesn't just seems like it, they confirmed like it is the most positive thing ever.
- TrainedMonkey 8mo ago
  It's software engineering crack. Starting a project feels amazing, features are shipping, a complex feature in the afternoon - ezpz. But AI lacks permanence, for every feature you start over from scratch, except there is more of codebase now, but the context window is still the same. So there is drift, codebase randomizes, edge cases proliferate, and the implementation velocity slows down. Full disclosure - I am a heavy codex user and I review and understand every line of code. I manually fight spurious tests it tries to add by pointing a similar one already exists and we can get coverage with +1 LOC vs +50. It's exhausting, but personal productivity is still way up. I think the future is bright because training / fine-tuning taste, dialing down agentic frameworks, introducing adversarial agents, and increasing model context windows all seem attainable and stackable.
  - tuhgdetzhh 8mo ago
    I think that the current test suite is far too small. For the Claude Code codebase, a sensible next step would be to generate thousands of tests. Without that kind of coverage, regressions are likely, and the existing checks and review process do not appear sufficient to reliably prevent them. My request is that an entirely LLM-written feature should only be eligible for merge once all of those generated tests pass, so we have objective evidence that the change preserves existing behavior.
  - kaydub 8mo ago
    I usually have multiple agents up working on a codebase. But it's typically 1 agent building out features and 1 or 2 agents code reviewing, finding code smells, bad architecture, duplicated code, stale/dead code, etc. I'm definitely faster, but there's a lot of LLM overhead to get things done right. I think if you're just using a single agent/session you're missing out on some of the speed gains. I think a lot of the gains I get using an LLM is because I can have the multiple different agent sessions work on different projects at the same time.
- MrDarcy 8mo ago
  I know at least one of the companies behind a coding agent we all have heard of has called in human experts to clean up their vibe coded IAC mess created in the last year.
klodolph 8mo ago
I’m happy to throw an LLM at our projects but we also spend time refactoring and reviewing each other’s code. When I look at the AI-generated code I can visualize the direction it’s headed in—lots of copy-pasted code with tedious manual checks for specific error conditions and little thought about how somebody reading it could be confident that the code is correct. I can’t understand how people would run agents 24/7. The agent is producing mediocre code and is bottlenecked on my review & fixes. I think I’m only marginally faster than I was without LLMs.
- gpm 8mo ago
  > with tedious manual checks for specific error conditions And specifically: Lots of checks for impossible error conditions - often then supplying an incorrect "default value" in the case of those error conditions which would result in completely wrong behavior that would be really hard to debug if a future change ever makes those branches actually reachable.
  - klodolph 8mo ago
    I always thought that the vast majority of your codebase, the right thing to do with an error is to propagate it. Either blindly, or by wrapping it with a bit of context info. I don’t know where the LLMs are picking up this paranoid tendency to handle every single error case. It’s worth knowing about the error cases, but it requires a lot more knowledge and reasoning about the current state of the program to think about how they should be handled. Not something you can figure out just by looking at a snippet.
    - stefan_ 8mo ago
      The answer (as usual) is reinforcement learning. They gave ten idiots some code snippets, and all of them went for the "belt and braces" approach. So now thats all we get, ever. It's like the previous versions that spammed emojis everywhere despite that not being a thing whatsoever in their training data. I don't think they ever fixed that, just put a "spare us the emojis" instruction in the system prompt bandaid.
    - 8mo ago
einpoklum 8mo ago
> When I investigated I found the docs and implementation are completely out of sync, but the implementation doesn’t work anyway. That is not an uncommon occurrence in human-written code as well :-\
- tobyjsullivan 8mo ago
  Someone said it best after one of those AWS outages from a fat-fingered config change: > Automation doesn't just allow you to create/fix things faster. It also allows you to break things faster. https://news.ycombinator.com/item?id=13775966 https://news.ycombinator.com/item?id=13775966 Edit: found the original comment from NikolaeVarius
data_ders 8mo ago
omg are you me? I had this exact same problem last week
nrds 8mo ago
What else could they do? If they don't vibecode Claude Code it is a bad look.

by eunoia 8mo ago

skerit 8mo ago

I switched to OpenCode, away from Claude-Code, because Claude-Code is _so_ buggy.

heliumtera 8mo ago

>Seems like everyone over there is vibing and no one is rationalizing the whole. Claude Code creator literally brags about running 10 agents in parallel 24/7. It doesn't just seems like it, they confirmed like it is the most positive thing ever.

TrainedMonkey 8mo ago
It's software engineering crack. Starting a project feels amazing, features are shipping, a complex feature in the afternoon - ezpz. But AI lacks permanence, for every feature you start over from scratch, except there is more of codebase now, but the context window is still the same. So there is drift, codebase randomizes, edge cases proliferate, and the implementation velocity slows down. Full disclosure - I am a heavy codex user and I review and understand every line of code. I manually fight spurious tests it tries to add by pointing a similar one already exists and we can get coverage with +1 LOC vs +50. It's exhausting, but personal productivity is still way up. I think the future is bright because training / fine-tuning taste, dialing down agentic frameworks, introducing adversarial agents, and increasing model context windows all seem attainable and stackable.
- tuhgdetzhh 8mo ago
  I think that the current test suite is far too small. For the Claude Code codebase, a sensible next step would be to generate thousands of tests. Without that kind of coverage, regressions are likely, and the existing checks and review process do not appear sufficient to reliably prevent them. My request is that an entirely LLM-written feature should only be eligible for merge once all of those generated tests pass, so we have objective evidence that the change preserves existing behavior.
- kaydub 8mo ago
  I usually have multiple agents up working on a codebase. But it's typically 1 agent building out features and 1 or 2 agents code reviewing, finding code smells, bad architecture, duplicated code, stale/dead code, etc. I'm definitely faster, but there's a lot of LLM overhead to get things done right. I think if you're just using a single agent/session you're missing out on some of the speed gains. I think a lot of the gains I get using an LLM is because I can have the multiple different agent sessions work on different projects at the same time.
MrDarcy 8mo ago
I know at least one of the companies behind a coding agent we all have heard of has called in human experts to clean up their vibe coded IAC mess created in the last year.

klodolph 8mo ago

I’m happy to throw an LLM at our projects but we also spend time refactoring and reviewing each other’s code. When I look at the AI-generated code I can visualize the direction it’s headed in—lots of copy-pasted code with tedious manual checks for specific error conditions and little thought about how somebody reading it could be confident that the code is correct. I can’t understand how people would run agents 24/7. The agent is producing mediocre code and is bottlenecked on my review & fixes. I think I’m only marginally faster than I was without LLMs.

gpm 8mo ago
> with tedious manual checks for specific error conditions And specifically: Lots of checks for impossible error conditions - often then supplying an incorrect "default value" in the case of those error conditions which would result in completely wrong behavior that would be really hard to debug if a future change ever makes those branches actually reachable.
- klodolph 8mo ago
  I always thought that the vast majority of your codebase, the right thing to do with an error is to propagate it. Either blindly, or by wrapping it with a bit of context info. I don’t know where the LLMs are picking up this paranoid tendency to handle every single error case. It’s worth knowing about the error cases, but it requires a lot more knowledge and reasoning about the current state of the program to think about how they should be handled. Not something you can figure out just by looking at a snippet.
  - stefan_ 8mo ago
    The answer (as usual) is reinforcement learning. They gave ten idiots some code snippets, and all of them went for the "belt and braces" approach. So now thats all we get, ever. It's like the previous versions that spammed emojis everywhere despite that not being a thing whatsoever in their training data. I don't think they ever fixed that, just put a "spare us the emojis" instruction in the system prompt bandaid.
  - 8mo ago

einpoklum 8mo ago

> When I investigated I found the docs and implementation are completely out of sync, but the implementation doesn’t work anyway. That is not an uncommon occurrence in human-written code as well :-\

tobyjsullivan 8mo ago
Someone said it best after one of those AWS outages from a fat-fingered config change: > Automation doesn't just allow you to create/fix things faster. It also allows you to break things faster. https://news.ycombinator.com/item?id=13775966 https://news.ycombinator.com/item?id=13775966 Edit: found the original comment from NikolaeVarius

data_ders 8mo ago

omg are you me? I had this exact same problem last week

nrds 8mo ago

What else could they do? If they don't vibecode Claude Code it is a bad look.