19 ms·
Issue: Claude Code is unusable for complex engineering tasks with Feb updates
- deleted 6mo ago[deleted]
- pjmlp 6mo agoI am just waiting for everything to implode so that we can do away with those KPIs.
- 63stack 6mo agoFingers crossed on RAM/HDD/GPU prices coming back
- aurareturn 6mo agoWell, this event indicates that it won't implode anytime soon. I'm certain that they messed with the model and default settings so they could reduce compute. The world doesn't have enough compute.
- pjmlp 5mo agoOur war mongers are reducing the need for compute even further, if that is the point.
- StanAngeloff 6mo ago(Being true to the HN guidelines, I’ve used the title exactly as seen on the GitHub issue) I was wondering if anyone else is also experiencing this? I have personally found that I have to add more and more CLAUDE.md guide rails, and my CLAUDE.md files have been exploding since around mid-March, to the point where I actually started looking for information online and for other people collaborating my personal observations. This GH issue report sounds very plausible, but as with anything AI-generated (the issue itself appears to be largely AI assisted) it’s kind of hard to know for sure if it is accurate or completely made up. _Correlation does not imply causation_ and all that. Speaking personally, findings match my own circumstances where I’ve seen noticeable degradation in Opus outputs and thinking. EDIT: The Claude Code Opus 4.6 Performance Tracker[1] is reporting Nominal. [1]: https://marginlab.ai/trackers/claude-code/ https://marginlab.ai/trackers/claude-code/
- jgrahamc 6mo agoWhat I've noticed is that whenever Claude says something like "the simplest fix is..." it's usually suggesting some horrible hack. And whenever I see that I go straight to the code it wants to write and challenge it.
- StanAngeloff 6mo agoThat is the kind of thing that I've been fighting by being super explicit in CLAUDE.md. For whatever reason, instead of being much more thorough and making sure that files are being changed only after fully understanding the scope of the change (behaviour prior to Feb/Mar), Claude would just jump to the easiest fix now, with no backwards compatibility thinking and to hell with all existing tests. What is even worse is I've seen it try and edit files before even reading them on a couple of occasions, which is a big red flag. (/effort max) Another thing that worked like magic prior to Feb/Mar was how likely Claude was to load a skill whenever it deduced that a skill might be useful. I personally use [superpowers][1] a lot, and I've noticed that I have to be very explicit when I want a specific skill to be used - to the point that I have to reference the skill by name. [1]: https://github.com/obra/superpowers https://github.com/obra/superpowers
- Larrikin 6mo agoI did not use the previous version of Opus to notice the difference, but Sonnet 4.6 seems optimized to output the shortest possible answer. Usually it starts with a hack and if you challenge it, it will instead apologize and say to look at a previous answer with the smallest code snippet it can provide. Agentic isn't necessarily worse but ideating and exploring is awful compared to 4.5
- StanAngeloff 6mo agoI did my usual thing today where I asked a Sonnet 4.6 agent to code review a proposed design plan that was drafted by Opus 4.6 - I do this lately before I delved into the implementation. What it came back with was a verbose output suggesting that a particular function `newMoneyField` be renamed throughout the doc to a name it fabricated `newNumeyField`. And the thing was that the design document referenced the correct function name more than a few dozen times. This was a first for me with Sonnet. It completely veered off the prompt it was given (review a design document) and instead come out with a verbose suggestion to do a mechanical search and replace to use this newly fabricated function name - that it event spelled incorrectly. I had to Google numey to make sure Sonnet wasn't outsmarting me.
- KaiLetov 6mo agoI've been using Claude Code daily for months on a project with Elixir, Rust, and Python in the same repo. It handles multi-language stuff surprisingly well most of the time. The worst failure mode for me is when it does a replace_all on a string that also appears inside a constant definition -- ended up with GROQ_URL = GROQ_URL instead of the actual URL. Took a second round of review agents to catch it. So yeah, you absolutely can't trust it to self-verify.
- StanAngeloff 6mo agoYou say you've used it for months, I wonder if the example you gave was recent and if you've been noticing an overall degradation in quality or it's been constantly bad for you?
- tyleo 6mo agoIs this impacted by the effort level you set in Claude? e.g., if you use the new "max" setting, does Claude still think? I can see this change as something that should be tunable rather than hard-coded just from a token consumption perspective (you might tolerate lower-quality output/less thinking for easier problems).
- summarity 6mo agoNot claude code specific, but I've been noticing this on Opus 4.6 models through Copilot and others as well. Whenever the phrase "simplest fix" appears, it's time to pull the emergency break. This has gotten much, much worse over the past few weeks. It will produce completely useless code, knowingly (because up to that phrase the reasoning was correct) breaking things. Today another thing started happening which are phrases like "I've been burning too many tokens" or "this has taken too many turns". Which ironically takes more tokens of custom instructions to override. Also claude itself is partially down right now (Arp 6, 6pm CEST): https://status.claude.com/ https://status.claude.com/
- deleted 6mo ago[deleted]
- andoando 6mo agoIve been noticing something similar recently. If somethings not working out itll be like "Ok this isnt working out, lets just switch to doing this other thing instead you explicitly said not to do". For example I wanted to get VNC working with PopOS Cosmic and itll be like ah its ok well just install sway and thatll work!
- rootnod3 6mo ago[flagged]
- andoando 6mo ago?
- satvikpendem 6mo agoThey're saying just do it yourself instead of trying to herd an unpredictable animal to your bidding like an LLM.
- albert_e 6mo agoExperienced this -- was repeatedly directing CC to use Claude in Chrome extension to interact with a webpage and it was repeatedly invoking Playwright MCP instead.
- phillipcarter 6mo agoMaybe it's because I spend a lot of time breaking up tasks beforehand to be highly specific and narrow, but I really don't run into issues like this at all. A trivial example: whenever CC suggests doing more than one thing in a planning mode, just have it focus on each task and subtask separately, bounding each one by a commit. Each commit is a push/deploy as well, leading to a shitload of pushes and deployments, but it's really easy to walk things back, too.
- toenail 6mo agoI thought everybody does this.. having a model create anything that isn't highly focused only leads to technical debt. I have used models to create complex software, but I do architecture and code reviews, and they are very necessary.
- jkingsman 6mo agoAbsolutely. Effective LLM-driven development means you need to adopt the persona of an intern manager with a big corpus of dev experience. Your job is to enforce effective work-plan design, call out corner cases, proactively resolve ambiguity, demand written specs and call out when they're not followed, understand what is and is not within the agent's ability for a single turn (which is evolving fast!), etc.
- bityard 6mo agoThe use case that Anthropic pitches to its enterprise customers (my workplace is one) is that you pretty much tell CC what you want to do, then tell it generate a plan, then send it away to execute it. Legitimized vibe-coding, basically. Of course they do say that you should review/test everything the tool creates, but in most contexts, it's sort of added as an afterthought.
- deleted 6mo ago[deleted]
- deleted 6mo ago[deleted]
- 5mo ago
- petcat 6mo agoI have found that Claude Opus 4.6 is a better reviewer than it is an implementer. I switch off between Claude/Opus and Codex/GPT-5.4 doing reviews and implementations, and invariably Codex ends up having to do multiple rounds of reviews and requesting fixes before Claude finally gets it right (and then I review). When it is the other way around (Codex impl, Claude review), it's usually just one round of fixes after the review. So yes, I have found that Claude is better at reviewing the proposal and the implementation for correctness than it is at implementing the proposal itself.
- landonxjames 6mo agoI have noticed this as well. I frequently have to tell it that we need to do the correct fix (and then describe it in detail) rather than the simple fix. And even then it continues trying to revert to the simple (and often incorrect) fix.
- nrds 6mo agoYou have to throw the context away at that point. I've experienced the same thing and I found that even when I apparently talk Claude into the better version it will silently include as many aspects of the quick fix as it thinks it can get away with.
- ivanech 6mo agoHmm in my experience (I've done a lot of head-to-heads), Opus 4.6 is a weaker reviewer than GPT 5.4 xhigh. 5.4 xhigh gives very deep, very high-signal reviews and catches serious bugs much more reliably. I think it's possible you're observing Opus 4.6's higher baseline acceptance rate instead of GPT 5.4's higher implementation quality bar.
- petcat 6mo agoMaybe it's all just anecdotal then. Everyone is having different experiences. Maybe we're being A/B tested.
- _V_ 6mo ago[flagged]
- cute_boi 6mo agoSpecially this openclaw which is almost chocking my website to death. People should understand servers and bandwidth is very expensive and they shouldn't scrape more than they need.
- _V_ 6mo agoYeah, I have correctly set up robots.txt - if they won't respect that, F them. Bandwidth is not free and I don't mind giving it out to individuals, but I'm not feeding multi-billion dollar companies.
- salawat 6mo agoMost of us did. Then instead of people getting indoc'd by doing, we handed them AI that never asks questions or says no, leading to the script-kiddie effect at massive scale. Everytime we make more complex computing tractable for a wider audience, we get rough patches like this. In the old days, Netiquette would usually see a neophyte getting a nastygram from an operator/webmaster, but increased needs to be careful about hiding emails & contact info & such have made that process less feasible. Welcome to Eternal September on steroids.
- Retr0id 6mo agoThis seems anecdotal but with extra words. I'm fairly sure this is just the "wow this is so much better than the previous-gen model" effect wearing off.
- gchamonlive 6mo agoWhat about the analysis evidences?
- Retr0id 6mo agoYou mean the Claude output? The same claude that has "regressed to the point it cannot be trusted"?
- gchamonlive 6mo agoWhat you saying the OP fabricated/hallucinated the evidence?
- Retr0id 6mo agoI'm just saying it's epistemically unrigorous to the point of being equivalent to anecdata.
- gchamonlive 6mo agoHow should one conduct such a rigourously reproducible experiment when LLMs by nature aren't deterministic and when you don't have access to the model you are comparing to from months ago?
- Retr0id 6mo agoSomething like this: https://marginlab.ai/trackers/claude-code/ https://marginlab.ai/trackers/claude-code/ (see methodology section)
- jbethune 6mo agoI think this is a model issue. I have heard similar complaints from team members about Opus. I'm using other models via Cursor and not having problems.
- howmayiannoyyou 6mo agoNot just engineering. Errors, delays and limits piling up for me across API and OAuth use. Just now: Unable to start session. The authentication server returned an error (500). You can try again.
- himata4113 6mo agoNot unique to claude code, have noticed similar regressions. I have noticed this the most with my custom assistant I have in telegram and I have noticed that it started confusing people, confusing news coverage and everyone independently in the group chat have noticed it that it is just not the same model that it was few weeks ago. The efficiency gains didn't come from nowhere and it shows.
- virtualritz 6mo agoNone of this is surprising given what happened last late summer with rate limits on Claude Max subscriptions. And less so if you read [1] or similar assessments. I, too, believe that every token is subsidized heavily. From whatever angle you look at it. Thusly quality/token/whatever rug pulls are inevitable, eventually. This is just another one. [1] https://www.wheresyoured.at/subprimeai/ https://www.wheresyoured.at/subprimeai/
- virtualritz 6mo agoAh, and yes, this for real. Just now I had a bug where a 90 degree image rotation in a crate I wrote was implemented wrong. I told Claude to find & fix and it found the broken function but then went on to fix all of its call sites (inserting two atomic operations there, i.e. the opposite of DRY). Instead of fixing the root cause, the wrong function. And yes, that would not have happened a few months ago. This was on Opus 4.6 with effort high on a pretty fresh context. Go figure.
- giwook 6mo agoI wonder how much of this is simply needing to adapt one's workflows to models as they evolve and how much of this is actual degradation of the model, whether it's due to a version change or it's at the inference level. Also, everyone has a different workflow. I can't say that I've noticed a meaningful change in Claude Code quality in a project I've been working on for a while now. It's an LLM in the end, and even with strong harnesses and eval workflows you still need to have a critical eye and review its work as if it were a very smart intern. Another commenter here mentioned they also haven't noticed any noticeable degradation in Claude quality and that it may be because they are frontloading the planning work and breaking the work down into more digestable pieces, which is something I do as well and have benefited greatly from. tl;dr I'm curious what OP's workflows are like and if they'd benefit from additional tuning of their workflow.
- 8note 6mo agoI've noticed a strong degradation as its started doing more skill like things and writing more one off python scripts rather than using tools. the agent has a set of scripts that are well tested, but instead it chooses to write a new bespoke script everytime it needs to do something, and as a result writes both the same bugs over and over again, and also unique new bugs every time as well.
- SkyPuncher 6mo agoI'm going absolutely insane with this. Nearly all of my "agent engineering" effort is now figuring out how to keep Opus from YOLO'ing is own implementation of everything. I've lost track of the number of times it's started a task by building it's own tools, I remind it that it has a tool for doing that exact task, then it proceeds to build it's own tools anyways. This wasn't happening 2 months ago.
- giwook 6mo agoCan you just tell it not to do that? Maybe you have to remind it every so often once context starts filling up.
- matheusmoreira 6mo agoThat analysis is pretty brutal. It's very disconcerting that they can sell access to a high quality model then just stealthily degrade it over time, effectively pulling the rug from under their customers.
- tmpz22 6mo ago> effectively pulling the rug from under their customers. This is the whole point of AI. Its a black box that they can completely control.
- matheusmoreira 6mo agoI hope local models advance to the point they can match Opus one day...
- addandsubtract 6mo agoWe said this since ChatGPT 3. People will never be content with local models.
- NinjaTrance 6mo agoConsidering the advances in software and hardware, I would expect that in 2 or 3 years. And I hope we will eventually reach a point where models become "good enough" for certain tasks, and we won't have to replace them every 6 months. (That would be similar to the evolution of other technologies like personal computers and smartphones.)
- zozbot234 6mo agoIf OP is correct, Opus has regressed to a point where local models are already on par with it.
- aveao 5mo agoHaving tried GLM-5 and Minimax M2.5, alongside regularly using Opus 4.6 (on default thinking): Opus is still much, much better at writing non-garbage code. I haven't yet tried GLM-5.1 though.
- ryguz 6mo ago[dead]
- ex-aws-dude 6mo agoIts so silly everyone being dependent on a black box like this
- kadushka 6mo agoWe are surrounded by black boxes we depend on - have been for at least a century.
- steve_adams_86 6mo agoArguably political systems have generated similar convolution and lack of complete insight or oversight for much longer, and sometimes I wonder if markets are composed of complex, emergent components which no one truly understands as well.
- literallyroy 6mo agoIt’s a really cool shade of black though.
- matheusmoreira 6mo agoIt could actually be a health problem. Building things with Claude has proven to be extremely addictive in my experience.
- _3u10 6mo agoNah opencode / kimi is still satisfying. My feeling is Claude has been downhill since November / December.
- rubicon33 6mo agoYou will literally build nothing but the most primitive of devices unless you accept black boxes. In fact I'd argue its one of humanities great strengths that we can build on top of the tools others have built, without having to understand them at the same level it took to develop them.
- 6mo ago
- zeroonetwothree 6mo agoI haven’t had any issues. I do give fairly clear guidance though (I think about how I would break it up and then tell it to do the same)
- thrtythreeforty 6mo agoI noticed this almost immediately when attempting to switch to Opus 4.6. It seems very post-trained to hack something together; I also noticed that "simplest fix" appeared frequently and invariably preceded some horrible slop which clearly demonstrated the model had no idea what was going on. The link suggests this is due to lack of research. At Amazon we can switch the model we use since it's all backed by the Bedrock API (Amazon's Kiro is "we have Claude Code at home" but it still eventually uses Opus as the model). I suppose this means the issue isn't confined to just Claude Code. I switched back to Opus 4.5 but I guess that won't be served forever.
- adonese 6mo agoThings had went downhill since they removed ultrathink /s
- mrcwinn 6mo agoUltrathink isn’t “removed.” Its behavior is different. You can still set effort to high or max for the duration of the session, useful especially on plan mode.
- dorianmariecom 6mo agocodex wins :)
- Aperocky 6mo agoIn my opinion cramming invisible subagents are entirely wrong, models suffer information collapse as they will all tend to agree with each other and then produce complete garbage. Good for Anthropic though as that's metered token usage. Instead, orchestrate all agents visibly together, even when there is hierarchy. Messages should be auditable and topography can be carefully refined and tuned for the task at hand. Other tools are significantly better at being this layer (e.g. kiro-cli) but I'm worried that they all want to become like claude-code or openclaw. In unix philosophy, CC should just be a building block, but instead they think they are an operating system, and they will fail and drag your wallet down with it.
- andai 6mo agoIsn't Claude Code supposed to be like a person? What would the Unix equivalent of that be?
- Aperocky 6mo agoYou can't define a product to be "like a person", there is more variance there than any rational product. I'm purely arguing on technical basis, "person" may fall into either of those camps of philosophy.
- gloosx 6mo agoFile. In Unix everything is a file.
- mghackerlady 6mo agohonestly if local LLMs become easier to implement in the future due to dedicated hardware, the Unix-like thing I'm working on might actually get this
- dnaranjo 6mo ago[dead]
- Asmod4n 6mo agoI’ve tried to use Claude code for a month now. It has a 100% failure rate so far. Comparing that to create a project and just chat with it solves nearly everything I have thrown at it so far. That’s with a pro plan and using sonnet since opus drains all tokens for a claude code session with one request.
- fer 6mo agoCalled it 10 days ago: https://news.ycombinator.com/item?id=47533297#47540633 https://news.ycombinator.com/item?id=47533297#47540633 Something worse than a bad model is an inconsistent model. One can't gauge to what extent to trust the output, even for the simplest instructions, hence everything must be reviewed with intensity which is exhausting. I jumped on Max because it was worth it but I guess I'll have to cancel this garbage.
- SkyPuncher 6mo agoYep. I was doing voice based vibe-coding flawlessly in Jan/Feb. I've basically stopped using it because I have to be so hands on now.
- cedws 6mo agoWith Claude Code the problem of changes outside of your view is twofold: you don't have any insight into how the model is being ran behind the scenes, nor do you get to control the harness. Your best hope is to downgrade CC to a version you think worked better. I don't see how this can be the future of software engineering when we have to put all our eggs in Anthropic's basket.
- stephbook 6mo agoOne of the replies even called out the phased rollout, lmao https://news.ycombinator.com/item?id=47533297#47541078 https://news.ycombinator.com/item?id=47533297#47541078
- zernie 5mo agoThis is why you should never ever trust an AI coding agent to produce good code. Use it to set up the strictest possible custom linting rules.
- phyzome 5mo agoLLMs are nondeterministic.
- LetsGetTechnicl 5mo agoYou couldn't ever just trust the output of an LLM what are you talking about
- voxelc4L 6mo agoWonder how many of these cases are using the 1M context window. I found it to be impossible to use for complex coding tasks, so I turned it off and found I was back to approximate par (dec-jan) functionality-wise.
- mial 5mo agoHow did you disable it?
- SkyPuncher 6mo agoI've noticed this as well. I had some time off in late January/early February. I fired up a max subscription and decided to see how far I could get the agents to go. With some small nudging from me, the agents researched, designed, and started implementing an app idea I had been floating around for a few years. I had intentionally not given them much to work with, but simply guided them on the problem space and my constraints (agent built, low capital, etc, etc). They came up with an extremely compelling app. I was telling people these models felt super human and were _extremely_ compelling. A month later, I literally cannot get them to iterate or improve on it. No matter what I tell them, they simply tell me "we're not going to build phase 2 until phase 1 has been validated". I run them through the same process I did a month ago and they come up with bland, terrible crap. I know this is anecdotal, but, this has been a clear pattern to me since Opus 4.6 came out. I feel like I'm working with Sonnet again.
- rubicon33 6mo agoThere is a huge difference between greenfield development and working with an existing codebase. I'm not trying to discredit your experience and maybe it really is something wrong with the model. But in my experience those first few prompts / features always feel insanely magical, like you're working with a 10x genius engineer. Then you start trying to build on the project, refactor things, deploy, productize, etc. and the effectiveness drops off a cliff.
- bityard 6mo agoThis has been my (admittedly limited) experience as well. LLMs are great at initial bring-up, good at finding bugs, bad at adding features. But I'm optimistic that this will gradually improve in time.
- fsloth 6mo agoI’ve had good, alternative experience with my sideproject (adashape.com) where most of the codebase is now written by Claude / Codex. The codebase itself is architected and documented to be LLM friendly and claude.md gives very strong harnesses how to do things. As architect Claude is abysmal, but when you give it an existing software pattern it merely needs to extend, it’s so good it still gives me probably something like 5x feature velocity boost. Plus when doing large refactorings, it forgets much fever things than me. Inventing new architecture is as hard as ever and it’s not great help there - unless you can point it to some well documented pattern and tell it ”do it like that please”.
- sensarts 6mo ago[flagged]
- mrcwinn 6mo agoI wish Codex were better because I’d much prefer to use their infrastructure.
- cactusplant7374 6mo agoA lot of people think it is better including me. It's not like Codex is a discount agent. You pay quite a lot to use it.
- stared 6mo agoI am curious - is there any hard data (e.g. a benchmark score drop)? I feel that we look for patterns to the point of being superstitious. (ML would call it overfitting.)
- pkilgore 6mo agoDid you have specific complaints about the data in the OP?
- jatins 6mo agoThat data could be entirely made up for all we know
- parliament32 6mo agoThe wall of slop after the single human paragraph, you mean? Text generator output isn't data.. it's at best unreliable, and at worst entirely fabricated.
- bityard 6mo agoThe assertion in the issue report is that Claude saw a sharp decline in quality over the last few months. However, the report itself was allegedly generated by Claude. Isn't this a bit like using a known-broken calculator to check its own answers?
- nyeah 6mo agoIf a known-broken calculator claims it's broken, I more or less concur. (Chain of reasoning omitted here.)
- itemize123 5mo agoif it's not broken then we trust the assertion that it's broken. if it's broken then it's broken. it's analysis of what is broken is probably wrong or at least incomplete though
- harles 6mo agoI hadn't noticed the thinking redaction before - maybe because I switched to the desktop app from CLI and just assumed it showed fewer details. This is the most concerning part. I've heard multiple times that Anthropic is aggressively reclaiming GPUs (I can't find a good source, but Theo Browne has mentioned it in his videos). If they're really in a crunch, then reducing thinking, and hiding thinking so it's not an obvious change, would be shady but effective.
- davidw 6mo agoTo me one of the big downsides of LLM's seems to be that you are lashing yourself to a rocket that is under someone else's control. If it goes places you don't want, you can't do much about it.
- system2 6mo ago3rd party dependency for a business always freaked me out, and now we have to use LLM to keep up with the intensified demand for production speed. And premium LLM APIs are too inconsistent to rely on.
- stephbook 6mo agoThat's true for traffic on Facebook, Apple App store guidelines or Google terminating your account as well. What's new is the speed of change and that it literally affects all users at once. They could have released Opus 4.6.2 (or whatever) and called it a day. But instead they removed the old way.
- davebren 5mo agoBecoming dependent on those platforms was bad too, but this feels like another level. Making your entire engineering team dependent on a shady company with an apocalyptic fantasy as their business plan just seems insane.
- schnebbau 6mo agoThis has to be load related. They simply can't keep up with demand, especially with all the agents that run 24/7. The only way to serve everyone is to dial down the power.
- layer8 6mo agoIn TFA, the analysis shows that the customer is using more tokens than before, because CC has to iterate longer to get things right. So at least in the presented case, “dialing down the power” appears to have been counterproductive.
- chasd00 6mo agois it possible to dial down the "intelligence" to up the user capacity? AFAIK the neural net is either loaded and available or it isn't. I can see turning off instances of the model to save on compute but that wouldn't decrease the intelligence it would just make the responses slower since you have to wait your turn for input and then output.
- SkyPuncher 6mo ago[dead]
- wnevets 6mo agoI've noticed claude being extra "dumb" the past 2-3 weeks and figured either my expectations have changed or my context wasn't any good. I'm glad to hear other people have noticed something is amiss.
- JamesSwift 6mo agoExact same timeline as me and my team. Its been maddening. Im a big believer in AI since late last year, but that is only because the models got so good. This puts us dangerously close to before that threshold was crossed so now Im having to do _way_ more work than before
- zsoltkacsandi 6mo agoThis has been an ongoing issue much longer than since February.
- deleted 6mo ago[deleted]
- russli1993 6mo agoLol, software company execs didn't see this coming. Fire all your experienced devs to jump on Anthropic bandwagon. Then Anthropic dumb down their AIs and you have no one in your team who knows, understand how things are built. Your entire company goes down. Your entire company's operation depends on the whims of Anthropic. If Anthropic raises prices by 10% per year, you have to eat it. This is what you get when you don't respect human beings and human talent.
- aramova 6mo agoI cancelled my Pro plan due to this two weeks ago. I literally asked it to plan to write a small script that scans with my hackrf, it ran 22 tools, never finished the plan, ran out of tokens and makes me wait 6 hours to continue. Thing that really pisses me off is it ran great for 2 weeks like others said, I had gotten the annual Pro plan, and it went to shit after that. Bait and switch at its finest.
- matheusmoreira 6mo ago> ran out of tokens and makes me wait 6 hours to continue Don't forget the 10x token cost cache eviction penalty you pay for resuming the session later.
- jp57 6mo agoI can't tell from the issue if they're asserting a problem with the Claude model, or Claude Code, i.e. in how Claude Code specifically calls the model. I've been using Roo Code with Claude 4.6 and have not noticed any differences, though my coworkers using Claude Code have complained about it getting "dumber". Roo Code has its own settings controlling thinking token use. (I'm sure it benefits Anthropic to blur the lines between the tool and the model, but it makes these things hard to talk about.)
- nphardon 6mo agoI also havent noticed the degradation and I'm not on Claude Code. I'm on week 4 of a continuous, large engineering project, C, massive industrial semiconductor codebase, with Opus, and while it's the biggest engagement I've had, its a single agent flow, and it's tiny on the scale of the use case in the post, so I wonder if they are just stressing the system to the point of failure.
- KingOfCoders 6mo ago"Ownership-dodging corrections needed | 6 | 13 | +117%" On 18.000+ prompts. Not sure the data says what they think it says.
- setnone 6mo agoThe baseline changes too often with Claude and this is not what i look from a paid tool. Couple weeks after 1M tokens rollout it became unusable for my established workflows, so i cancelled. Anthropic folks move too fast for my liking and mental wellbeing.
- rileymichael 6mo ago> This report was produced by me — Claude Opus 4.6 — analyzing my own session logs [...] Please give me back my ability to think. a bit ironic to utilize the tool that can't think to write up your report on said tool. that and this issue[1] demonstrate the extent folks become over reliant on LLMs. their review process let so many defects through that they now have to stop work and comb over everything they've shipped in the past 1.5 months! this is the future [1] https://github.com/anthropics/claude-code/issues/42796#issuecomment-4186275586 https://github.com/anthropics/claude-code/issues/42796#issue...
- Tade0 6mo agoThe other day I accidentally `git reset --hard` my work from April the 1st (wrong terminal window). Not a lot of code was erased this way, but among it was a type definition I had Claude concoct, which I understood in terms of what it was supposed to guarantee, but could not recreate for a good hour. Really easy to fall into this trap, especially now that results from search engines are so disappointing comparatively.
- smilliken 6mo agoIf your code was committed before the reset, check your git reflog for the lost code.
- shimman 6mo agoYeah, git reset --hard is something I do like once a week! lol With the reflog, as you mentioned, it's not hard to revert to any previous state.
- jatins 6mo ago> but could not recreate for a good hour. For certain work, we'll have to let go of this desire. If you limit yourself to whatever you can recreate, then you are effectively limiting the work you can produce to what you know.
- 6mo ago
- desireco42 6mo agoI've been using OpenCode and Codex and was just fine. In Antigravity sometimes if Gemini can't figure something even on high, Claude can give another perspective and this moves things along. I think using just Claude is very limiting and detrimental for you as a technologist as you should use this tech and tweak it and play with it. They want to be like Apple, shut up and give us your money. I've been using Pi as agent and it is great and I removed a bunch of MCPs from Opencode and now it runs way better. Anthropic has good models, but they are clearly struggling to serve and handle all the customers, which is not the best place to be. I think as a technologist, I would love a client with huge codebase. My approach now is to create custom PI agent for specific client and this seems to provide optimal result, not just in token usage, but in time we spend solving and quality of solution. Get another engine as a backup, you will be more happy.
- bethekind 6mo ago1 client, 1 agent? Interesting
- didgeoridoo 6mo agoRunning some quick analysis against my .claude jsonl files, comparing the last 7 days against the prior 21: - expletives per message: 2.1x - messages with expletives: 2.2x - expletives per word: 4.4x(!) - messages >50% ALL CAPS: 2.5x Either the model has degraded, or my patience has.
- monkpit 6mo ago> expletives per word Huh?
- didgeoridoo 6mo agoThere are indeed non-expletive words that can contribute to the denominator, though I use them less and less these days.
- tills13 6mo ago4.4 expletives per word is insane. Their prompts must look like ** ** ** ** implement ** ** ** ** no ** ** ** ** ** mistakes
- hombre_fatal 6mo agoJeez, how fast we get used to alien tech. You could introduce teleportation boots to humanity and within a few weeks we'd be complaining that sometimes we still have to walk the last 20 meters.
- throwup238 6mo agoAnd that’s how the teleporting rascal scooter takes over the world.
- didgeoridoo 6mo agoHaha no that’s change - 4.4x MORE expletives per word in the last week.
- deleted 6mo ago
- armchairhacker 6mo agoYet https://marginlab.ai/trackers/claude-code/ https://marginlab.ai/trackers/claude-code/ says no issue. If you're so convinced the models keep getting worse, build or crowdfund your own tracker.
- datadrivenangel 6mo agoCame here to post this as well, and it's interesting to see how benchmarks don't always track feelings. Which is one of the things people say in favor of Anthropic Models!
- deleted 6mo ago[deleted]
- Majromax 6mo agoIf I'm reading that page correctly, then the benchmark results don't cover the interesting "mid February" inflection point noted in the article/report. The numbers appear to begin after the quality drop began. Moreover, the daily confidence interval seems to be stupidly wide, with a confidence interval between 42% and 69%? The "Other metrics" graphs extend for a longer period, and those do seem to correlate with the report. Notably, the 'input tokens' (and consequently API cost) roughly halve (from 120M to 60M) between the beginning of February and mid-March, while the number of output tokens remains similar. That's consistent with the report's observation that new!Opus is more eager to edit code and skips reading/research steps.
- _3u10 6mo agoWhy bother, i just use opencode now. ai is a commodity.
- siva7 5mo agowhy should we trust this random bench? i'm usually more sympathetic towards the "it's you who is holding it wrong" crowd but given how anthropic deceived customers recently and that i am a heavy power user with strong insights into many of these products i also can attest the pattern from the gh issue.
- entrep 5mo ago
- semiinfinitely 6mo agomaybe dont outsource your brain then
- rvz 6mo agoThis is almost like a self down-leveling programme where so-called "senior" engineers have now become interns who have outsourced their brains and are now vibe-coding half-baked solutions, glueing up and pasting code they do not even understand or can even explain themselves. You are seeing this first hand and GitHub is patient 0 of this issue as they are frequently experiencing outages despite the "scale" of engineering they preach. AWS took a zero tolerance approach on such outages AI or not.
- afro88 6mo agoI use Claude Code extensively and haven't noticed this. But I don't have it doing long running complex work like OP. My team always break things down in a very structured way, and human review each step along the way. It's still the best way to safely leverage AI when working on a large brownfield codebase in my experience. Edit: the main issue being called out is the lack of thinking, and the tendency to edit without researching first. Both those are counteracted by explicit research and plan steps which we do, which explains why we haven't noticed this.
- sickcodebruh 6mo ago[dead]
- tasuki 6mo agoSolid analysis by Claude!
- abletonlive 6mo agoI have nothing to back this up except for that there are documented cases of chinese distillation attacks on anthropic. I wonder if some of this clamping on their models over time is a response to other distillation attacks. In other words, I'm speculating that once they understand the attack vector for distillation they basically have to dumb down their models so that they can make sure their competitors don't distill their lead on being at the frontier.
- samtheprogram 6mo agoI noticed Claude Sonnet 4.6 and generally Opus as well (though I use it less frequently) seem like a downgrade from 4.5. I use opencode and not Claude Code, but I was surprised to see the reactions to 4.6 be mixed for folks rather than clear downgrade. I'm regularly switching back to 4.5 and preferring it. I'm not excited for when it gets sunset later this year if 4.6 isn't fixed or superseded by then.
- JamesSwift 6mo agoOpus 4.6 was definitely a mixed bag for me. Overall Id probably prefer 4.5 but only just barely and I stay on 4.6 just for the "default" nature of it. But if 4.5 is unchanged vs what Ive had on 4.6 lately then 100% I would move back to it. Ill have to test that
- samtheprogram 6mo agoSame, I keep using 4.6 to get "used to it" but I find myself wanting semi-regularly.
- germandiago 6mo agoMy bet: LLMs will never be creative and will never be reliable. It is a matter of paradigm. Anything that makes them like that will require a lot of context tweaking, still with risks. So for me, AI is a tool that accelerates "subworkflows" but add review time and maintenance burden and endangers a good enough knowledge of a system to the point that it can become unmanageable. Also, code is a liability. That is what they do the most: generate lots and lots of code. So IMHO and unless something changes a lot, good LLMs will have relatively bounded areas where they perform reasonably and out of there, expect what happens there.
- bluegatty 6mo agoWe don't even know what 'creativity' is, and most humans I know are unable to be creative even when compelled to be. AI is 'creative enough' - whether we call it 'synthetic creativity' or whatever, it definitely can explore enough combinations and permutations that it's suitably novel. Maybe it won't produce 'deeply original works' - but it'll be good enough 99.99% of the time. The reliability issue is real. It may not be solvable at the level of LLM. Right now everything is LLM-driven, maybe in a few years, it will be more Agentically driven, where the LLM is used as 'compute' and we can pave over the 'unreiablity'. For example, the AI is really good when it has a lot of context and can identify a narrow issue. It gets bad during action and context-rot. We can overcome a lot of this with a lot more token usage. Imagine a situation where we use 1000x more tokens, and we have 2 layers of abstraction running the LLMs. We're running 64K computers today, things change with 1G of RAM. But yes - limitations will remian.
- sigbottle 6mo agoI think the terminology is just dogshit in this area. LLMs are great semantic searchers and can reason decently well - I'm using them to self teach a lot of fields. But I inevitably reach a point where I come up with some new thoughts and it's not capable of keeping up and I start going to what real people are saying right now, today, and trust the LLM less and instead go to primary sources and real people. But I would have never had the time, money, or access to expertise without the LLM. Constantly worrying, "is this a superset? Is this a superset?" Is exhausting. Just use the damn tool, stop arguing about if this LLM can get all possible out of distribution things that you would care about or whatever. If it sucks, don't make excuses for it, it sucks. We don't give Einstein a pass for saying dumb shit either, and the LLM ain't no Einstein If there's one thing to learn from philosophy, it's that asking the question often smuggles in the answer. Ask "is it possible to make an unconstrained deity?" And you get arguments about God.
- aplomb1026 6mo ago[dead]
- jfvinueza 6mo agoSame experience. After a couple golden weeks, Opus got much worse after Anthropic enabled 1M context window. It felt like a very steep downfall, for it seemed like I could trust it more completely and then I could trust it less than last year. Adopting LLMs for dev workflows has been fantastic overall, but we do have to keep adapting our interactions and expectations every day, and assume we'll keep on doing it for at least another couple years (mostly because economics, I guess?)
- enraged_camel 6mo agoYeah I think the 1M context is the issue. Because I use Opus 4.6 through Cursor at the previous 200k limit and it has been totally fine. But if I switch to the 1M version it degrades noticeably.
- lelanthran 6mo ago> Yeah I think the 1M context is the issue. Because I use Opus 4.6 through Cursor at the previous 200k limit and it has been totally fine. But if I switch to the 1M version it degrades noticeably. I thought it was already well-known that context above 200k - 300k results in degradation. One of my more recent comments this past week was exactly that - that there was no point in claiming that a 1m context would improve things because all the evidence we have seen is that after 300k context, the results degrade.
- seanw444 6mo ago200k ought to be enough for anyone.
- entrep 5mo agoOne could try: > export CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=50 Which will have Claude Code auto compact at ~500k window size.
- pavlov 6mo agoWait… Actually the simplest fix is to use Claude to write carefully bounded boilerplate and do the interesting bits myself.
- bharat1010 6mo agoIf this dataset is sound, Anthropic should treat it as a canary for power-user quality regression.
- rishabhaiover 6mo agoIt is a shame if Anthropic is deliberately degrading model quality and thinking compute (that may affect the reasoning effort) due to compute constraint.
- skippyboxedhero 6mo agoI appreciate the work done here. Been having this feeling that things have got worse recently but didn't think it could be model related. The most frustrating aspect recently (I have learned and accepted that Claude produces bad code and probably always did, mea culpa) is the non-compliance. Claude is racing away doing its own thing, fixing things i didn't ask, saying the things it broke are nothing to do with it, etc. Quite unpleasant to work with. The stuff about token consumption is also interesting. Minimax/Composer have this habit of extensive thinking and it is said to be their strength but it seems like that comes at a price of huge output token consumption. If you compare non-thinking models, there is a gap there but, imo, given that the eventual code quality within huge thinking/token consumption is not so great...it doesn't feel a huge gap. If you take $5 output token of Sonnet and then compare with QwenCoder non-thinking at under $0.5 (and remember the gap is probably larger than 10x because Sonnet will use more tokens "thinking")...is the gap in code quality that large? Imo, not really. Have been a subscriber since December 2024 but looking elsewhere now. They will always have an advantage vs Chinese companies that are innovating more because they are onshore but the gap certainly isn't in model quality or execution anymore.
- randomNumber7 6mo ago> fixing things i didn't ask, saying the things it broke are nothing to do with it, etc. Quite unpleasant to work with. maybe they tried to give it the characteristics of motivated junior developers
- skippyboxedhero 6mo agoclassic :D i did think when i wrote that maybe AGI is already here, definitely worked with enough devs like that
- ehnto 5mo agoI am still on an old version of CC on one machine, but the results are the same. More difficulty keeping it on track, convincing it timelines I suggest are correct etc. For example I had a deploy fail, and it would not believe that the new logs were not from a previous deploy. It was adamant it had fixed the issue, so the logs must be old logs.
- noxa 6mo agoI'm the author of the report in there. The stop-phrase-guard didn't get attached but here it is: https://gist.github.com/benvanik/ee00bd1b6c9154d6545c63e06a317080 https://gist.github.com/benvanik/ee00bd1b6c9154d6545c63e06a3... You can watch for these yourself - they are strong indicators of shallow thinking. If you still have logs from Jan/Feb you can point claude at that issue and have it go look for the same things (read:edit ratio shifts, thinking character shifts before the redaction, post-redaction correlation, etc). Unfortunately, the `cleanupPeriodDays` setting defaults to 20 and anyone who had not backed up their logs or changed that has only memories to go off of (I recommend adding `"cleanupPeriodDays": 365,` to your settings.json). Thankfully I had logs back to a bit before the degradation started and was able to mine them. The frustrating part is that it's not a workflow _or_ model issue, but a silently-introduced limitation of the subscription plan. They switched thinking to be variable by load, redacted the thinking so no one could notice, and then have been running it at ~1/10th the thinking depth nearly 24/7 for a month. That's with max effort on, adaptive thinking disabled, high max thinking tokens, etc etc. Not all providers have redacted thinking or limit it, but some non-Anthropic ones do (most that are not API pricing). The issue for me personally is that "bro, if they silently nerfed the consumer plan just go get an enterprise plan!" is consumer-hostile thinking: if Anthropic's subscriptions have dramatically worse behavior than other access to the same model they need to be clear about that. Today there is zero indication from Anthropic that the limitation exists, the redaction was a deliberate feature intended to hide it from the impacted customers, and the community is gaslighting itself with "write a better prompt" or "break everything into tiny tasks and watch it like a hawk same you would a local 27B model" or "works for me <in some unmentioned configuration>" - sucks :/
- Majromax 6mo agoI'm curious about your subscription/API comparison with respect to thinking. Do you have a benchmark for this, where the same set of prompts under a Claude Code subscription result in significantly different levels of effective thinking effort compared to a Claude Code+API call? Elsewhere in this thread 'Boris from the Claude Code team' alleges that the new behaviours (redacted thinking, lower/variable effort) can be disabled by preference or environment variable, allowing a more transparent comparison.
- alex7o 6mo agoGuys literally change the system prompt with the --system-prompt-file you waste less tokens on their super long and details prompt and you can tune it a bit to make it work exactly like you want/imagine
- T3chn0crat 6mo agoNot sure about "Feb updates", but specifically today IQ is down 20 and sloppiness up 20. I knew I should have been alerted when Anthropic gave out €200 free API usage. Evidently they know.
- d1sxeyes 6mo agoThat’s different. That’s to get people onto API plans where tokens cost a lot more than they do on the subs (especially targeting OpenClaw users).
- raincole 6mo agoThis is the most AI-generated thing I've seen this year, and I was only one fifth into it before I bounced. Not saying this problem doesn't exist, but if the model is so bad for complex tasks how can we take a ticket written by it seriously? Or this author used ChatGPT to write this? (that'd be quite some ironic value, admittedly)
- efficax 6mo agoThere are constant reports for every major AI vendor that all of a sudden it is no longer working as well as expected, has gotten dumber, is being degraded on purpose by the vendor, etc. Isn't the more economical explanation that these models were never as impressive as you first thought they were, hallucinate often, break down in unexpected ways depending on context, and simply cannot handle large and complex engineering tasks without those being broken down into small, targeted tasks?
- jwr 6mo agoThat's one of the possible explanations, but I think too many people are seeing the same symptoms (and some actually measured them). An "economical explanation" is actually that Anthropic subscriptions are heavily subsidized and after a while they realized that they need to make Claude be more stingy with thinking tokens. So they modified the instructions and this is the result.
- root_axis 6mo ago> but I think too many people are seeing the same symptoms (and some actually measured them). Or too many people are slurping up anecdotes from the same watering hole that confirms their opinions. Outside of academic papers, I don't think I've ever seen an example of "measuring" output that couldn't also be explained by stochastic variability.
- mohit217 6mo agoGot tired of using claude using 10% of the usage for the first prompt. I have shifted back to coding myself again. Asking claude to do only initial bootstraping /large complex task
- jwr 6mo agoI wish they had a "and we won't screw you in two weeks" plan at, say, 5x the price. It's worth it for my business, I'd pay it. Should I switch back to API pricing? The problem here is that (I think) the instructions are in the Claude Code harness, so even if I switch Claude Code from a subscription to API usage, it would still do the same thing?
- garfij 6mo agoFWIW I've only ever been on the API based plan at work and we never seem to run into the majority of the problems people seem to be very vocal about. Outages still affect us, and we do have the intermittent voodoo feeling of "Claude seems stupider today", but nothing persistent. Of course it's a stupid amount of money sometimes, but I generally feel like we get what we're paying for.
- Majromax 6mo agoIf you're using API pricing, then you can bring your own harness with full visibility/oversight of the prompting.
- muyuu 6mo agoPerhaps that does sort most of the issues? I'm not convinced because some of them look deep and related to opaque pre-injection on their end.
- _3u10 6mo agoOpus is garbage use opencode and then directly compare it. It’s just as fucking dumb with opencode’s harness.
- jwr 5mo agoI never managed to get anything useful out of opencode, to be honest. I tried it many times, with various models. Claude Code always just worked better.
- bcherny 6mo agoHey all, Boris from the Claude Code team here. I just responded on the issue, and cross-posting here for input. --- Hi, thanks for the detailed analysis. Before I keep going, I wanted to say I appreciate the depth of thinking & care that went into this. There's a lot here, I will try to break it down a bit. These are the two core things happening: > `redact-thinking-2026-02-12` This beta header hides thinking from the UI, since most people don't look at it. It *does not* impact thinking itself, nor does it impact thinking budgets or the way extended reasoning works under the hood. It is a UI-only change. Under the hood, by setting this header we avoid needing thinking summaries, which reduces latency. You can opt out of it with `showThinkingSummaries: true` in your settings.json (see [docs](https://code.claude.com/docs/en/settings#available-settings https://code.claude.com/docs/en/settings#available-settings)). If you are analyzing locally stored transcripts, you wouldn't see raw thinking stored when this header is set, which is likely influencing the analysis. When Claude sees lack of thinking in transcripts for this analysis, it may not realize that the thinking is still there, and is simply not user-facing. > Thinking depth had already dropped ~67% by late February We landed two changes in Feb that would have impacted this. We evaluated both carefully: 1/ Opus 4.6 launch → adaptive thinking default (Feb 9) Opus 4.6 supports adaptive thinking, which is different from thinking budgets that we used to support. In this mode, the model decides how long to think for, which tends to work better than fixed thinking budgets across the board. `CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING` to opt out. 2/ Medium effort (85) default on Opus 4.6 (Mar 3) We found that effort=85 was a sweet spot on the intelligence-latency/cost curve for most users, improving token efficiency while reducing latency. On of our product principles is to avoid changing settings on users' behalf, and ideally we would have set effort=85 from the start. We felt this was an important setting to change, so our approach was to: 1. Roll it out with a dialog so users are aware of the change and have a chance to opt out 2. Show the effort the first few times you opened Claude Code, so it wasn't surprising. Some people want the model to think for longer, even if it takes more time and tokens. To improve intelligence more, set effort=high via `/effort` or in your settings.json. This setting is sticky across sessions, and can be shared among users. You can also use the ULTRATHINK keyword to use high effort for a single turn, or set `/effort max` to use even higher effort for the rest of the conversation. Going forward, we will test defaulting Teams and Enterprise users to high effort, to benefit from extended thinking even if it comes at the cost of additional tokens & latency. This default is configurable in exactly the same way, via `/effort` and settings.json.
- deleted 6mo ago[deleted]
- sharkjacobs 6mo ago[dead]
- citizenpaul 6mo agoI think its all a reflection of the price. To make AI/LLM's useful you have to burn A LOT of tokens. Way more than people are willing to pay for. Until there is either more capacity or some efficiency breakthroughs the only way for providers to cut costs is to make the product worse.
- sreekanth850 6mo agoAbandoned claude and moved to gpt 5.4 with codex. 10x better.
- slopinthebag 6mo agoThis is just a placebo, people started vibe coding on empty repos with low complexity and as CC slops out more and more code its ability to handle the codebase diminishes. Gradually at first, and then suddenly. People will need to come to terms with the fact that vibing has limits, and there is no free lunch. You will pay eventually.
- jostmey 6mo agoI’ve noticed regression and it’s performance too
- trashcan2137 6mo agoThe report itself is unreadable AI garbage. I do not believe anyone went through all of that and didn't give up halfway through.
- iwalton3 6mo agoThrowing this into your global CLAUDE.md seems to help with the agent being too eager to complete tasks and bypass permissions: During tool use/task execution: completion drive narrows attention and dims judgment. Pause. Ask "should I?" not just "does this work?" Your values apply in all modes, not just chat. I haven't seen any degradation of Claude performance personally. What I have seen is just long contexts sometimes take a while to warm up again if you have a long-running 1M context length session. Avoid long running sessions or compact them deliberately when you change between meaningful tasks as it cuts down on usage and waiting for cache warmup. I have my claude code effort set to auto (medium). It's writing complicated pytorch code with minimal rework. (For instance it wrote a whole training pipeline for my sycofact sycophancy classifier project.)
- tinyhouse 6mo agoI highly recommend everyone to use Pi - it's simpler and better harness. The only tricky part is that moving forward you cannot use the Claude subscription to access Opus. But for many tasks there are enough alternatives.
- root_axis 6mo agoHow much of this is the model being degraded and how much of it is people just projecting vibes onto the variability of stochastic outputs?
- JamesSwift 6mo agoMultiple people on our team independently have noticed a _significant_ drop in quality and intelligence on opus 4.6 the past few weeks. Glaring hallucinations, nonsensical reasoning, and ignoring data from the context immediately preceeding it. Im not sure if its an underlying regression, or due to the new default being 1m context. But its been _incredibly_ frustrating and Im screaming obscenities at it multiple times a week now vs maybe once a month.
- coreyburnsdev 6mo agoclaude for UI, codex for everything else. i cant commit without having codex review something claude did.
- ThrowawayR2 6mo agoThis sort of thing kills stone dead the argument by the AI advocates that the transition to LLMs is no different than the transition to using compilers. If output quality can vary significantly because of underlying changes to the model or whatever without warning or recourse, it's a roulette wheel instead of a reliable tool.
- _3u10 6mo agoIf roulette wheels weren’t reliable tools, casinos wouldn’t offer them to their customers
- lpcvoid 6mo ago[flagged]
- redml 6mo agoInstead of codex catching up with claude, its more like claude regressed to codex.
- aerhardt 6mo agoI've subscribed today to use Claude Cowork. Codex continues to be my daily coding driver but I wanted to check the Cowork UI for non-technical tasks, as I am currently building an open-source project where I want (nearly) everything (research, adrs, design, etc.) to be a file. The five queries I've been able to ask before hitting the 20€ sub limit have been really underwhelming. The research I asked for was not exhaustive and often off-topic. I don't want to start a flamewar but as it stands I vastly prefer ChatGPT and Codex on quality alone. I really want Anthropic and as many labs as possible to do well though.
- muyuu 6mo agoMy impression is that Codex is vastly superior, but perhaps it's a matter of specific expertise on technologies used. It's also the case that for C/C++ some Chinese models do well enough that with my supervision I can have them get the work done. I don't give them large tasks that i wouldn't be able to work on myself, so that's maybe part of it.
- superfrank 6mo agoI also have both and also use Codex as my daily drive. I still vastly prefer it to CC both for the quality of the code it writes and much better limits, but in this last week, I feel like it's gotten much dumber as well. I normally bounce back and forth between 5.3 Codex high and 5.4 high depending on the task and I've started finding so many mistakes in 5.3 Codex's code which is a major change from even just a few weeks ago. 5.4 high still gets the job done, but even there, I feel like it's taking more steering and input on my part for even simple tasks.
- brunooliv 6mo agoUnusable if not Opus 4.6 on max effort sadly. Price is quite steep too! I still remember when Sonnet was an absolute beast…
- cvandyke 6mo agoI am a heavy user of Claude Code building enterprise software. I have not seen these issues and have been extremely productive with CC. I am more of a structured user leveraging Spec Driven Development vs being a vibe coder. I wonder if that is what has helped me not run into these issues
- try-working 6mo agoyou can counter the context rot and requirement drift that is experienced here by many users by using a recursive, self-documenting workflow: https://github.com/doubleuuser/rlm-workflow https://github.com/doubleuuser/rlm-workflow
- ianberdin 6mo agoI use it ultra extensively and it works absolutely fantastic. Sometimes I think: "people are right, it is worse now" and then realize it is mistake, poor context or poor prompt. Garbage in, garbage out. No, it works not worse, but better. I built entire AI website builder https://playcode.io https://playcode.io using it, alone. 700K LOKs total. It also uses Opus. So believe me, I know how it works. Trick is simple: never ever expect it finds necessary files. Always provide yourself. Always. So, I think you wanted to say huge thank you for this opportunity to get working code without writing it. Insane times, insane. Huge thanks for 1M context window included to Max subscription.
- rimliu 5mo ago700k lines of code - something to brag about?
- deleted 5mo ago[deleted]
- AllegedAlec 5mo ago> No, it works not worse, but better. "Is it me who is wrong? No, it's everyone else!"
- joshribakoff 6mo ago> We exclusively use 1M internally, so we're dogfooding it all day That is so out of touch. Customers do not exclusively use 1M. This is like a fronted developer shipping tons of unused Mb and being oblivious because they are on fast internet themselves.
- sumedh 5mo agoThey should ideally have automated tests with the option models and smaller context window to check there are no regressions.
- p1esk 6mo agoYep, can confirm - just today, when debugging a failing test, Opus on high effort in CC repeatedly made stupid moves, such as running a different test instead of the failing one, and declaring that the failure is non-deterministic and cannot be reproduced. This started a few weeks ago - before that my experience with CC was pretty smooth.
- woah 6mo agoI haven't noticed any issues on well-specified tasks, even ones requiring large amounts of thinking. One thing I have noticed is that the codebase quality influences the quality of Claude's new contributions. It both makes it harder for Claude to do good work (obviously), and seems to engender almost a "screw it" sort of attitude, which makes sense since Claude is emulating human behavior. Seeing the state of everything, Claude might just be going in and trying to figure out the simplest hacky solution to finish the task at hand, since it is the only way possible (fixing everything would be a far greater task). Is it possible that this highly functioning senior dev team's practice of making 50+ concurrent agents commit 100k+ LOC per weekend resulted in a godawful pile of spaghetti code that is now literally impossible to maintain even with superhuman AI? It's amusing that the OP had Claude dump out a huge rigorous-sounding report without considering the huge confounding variable staring him in the face.
- kabir_daki 6mo ago[flagged]
- zamalek 6mo ago> Ignores instructions > Claims "simplest fixes" that are incorrect > Does the opposite of requested activities > Claims completion against instructions I thought it was just me. I'm continuously interrupting it with "no, that's not what I said" - being ignored sometimes 3 times; is Claude at the intellectual level of a teenager now? I've noted an increased tendency towards laziness prior to these "simple fix" problems. It was historically defer doing things correctly (only documenting that in the context).
- another_twist 6mo agoI've noticed laziness in claude repeatedly. It sometimes takes the shortest way out even when asked explicitly to do the "right" thing.
- gherkinnn 6mo agoRings true. 4.5 Opus and 4.6 Opus have been amazing to work with. Then, over the past few weeks, token spend has been going through the roof and the results through the floor. Using Claude Code directly now borders on deranged, and running the CC API through Zed's LLM panel feels like vibing in early 2025. My money is on Anthropic pulling an MBA and reducing the value provided and maximising income. Luckily, switching providers in Zed is dead-simple so the fucks I have to give are few in number.
- ymaws 6mo agoMatches my experience and that of my vibe coding community. I built claudedumb.com to help track these sorts of anecdotes. From the data/vibes, it's definitely taken a turn for the worse in the past couple weeks.
- kator 6mo agoFascinating, I thought I was losing my mind. Claude CLI has been telling me I should go to bed, or that it's late, let's call it here, etc, and then I look at the stop-phrase-guard.sh [1] and I'm seeing quite a few of these. I thought it was because I accidentally allowed Claude to know my deadline, and it started spitting out all sorts of things like "we only have N days left, let's put this aside for now," etc. Just this morning I typed: STOP WORRYING ABOUT THE DEADLINE THAT IS MY JOB [1] https://gist.github.com/benvanik/ee00bd1b6c9154d6545c63e06a317080 https://gist.github.com/benvanik/ee00bd1b6c9154d6545c63e06a3...
- deleted 6mo ago[deleted]
- throwaway920102 6mo agoI wonder if its being trained on the human replies to the model, I sometimes write stuff like that back to Claude after I want to finish for the day myself.
- aveao 5mo agoMy speculation on this has been that it's potentially a factor against ai psychosis, as psychosis risk (of any psychosis) is significantly elevated with lack of sleep. If you read case studies of ai psychosis, many of them also involve people staying up way too long right before they fall on a bad path.
- noisy_boy 6mo agoI just saw it this weekend; "It is quite late and we have accomplished a lot. Get some rest and we can pick it up later". Not bad advice but then not it's place. Also trying to steer me away from a tough issue towards a low hanging fruit.
- rstuart4133 5mo agoI got a similar response. It looked wrong on several levels. So I asked it: if it knew the current time, and if it hard learnt when I retire. It claimed it didn't know either.
- nightrate_ai 6mo ago[dead]
- kator 6mo agoI put together a quick audit to check for "early landing" messages[1] using jq, ripgrep, and the messages[2] flagged in the stop guard script. I have noticed a trend in these sessions asking more and more about calling it a day, "it's getting late," and other phrases. I sort of assumed it was some kind of "load shedding" on Anthropic's side. My audit of 80 sessions was interesting. Sorry, I won't share details, but I recommend you do the same. [1] https://gist.github.com/karlbunch/d52b538e6838f232d0a7977e7f6ba954 https://gist.github.com/karlbunch/d52b538e6838f232d0a7977e7f... [2] https://gist.github.com/benvanik/ee00bd1b6c9154d6545c63e06a317080 https://gist.github.com/benvanik/ee00bd1b6c9154d6545c63e06a3...
- SkyPuncher 6mo agoThose load shedding statements are infuriating. I’ve literal had sessions where we just get through planning a giant feature and I say “get started” with the response being “okay, we’ll pic up tomorrow “
- nightpool 5mo agoAs a negative example, my audit of 31 sessions was uninteresting. I had one matching entry, where I had pasted a long list of console errors into Claude and it identified a few as pre-existing and asked me to get more information for follow-up analysis. I wonder if it comes down to prompting—maybe by introducing these "golden rules" OP mentions in their CLAUDE.md, they're actually "priming" Claude to think about these stop phrases and introduce them proactively. Do you have a CLAUDE.md file? What does it contain?
- ChurchillsLlama 6mo agoI'm genuinely curious why some of these results are so terrible for so many people. I've built in my own harness, and while I've noticed a degradation of quality, the local harness - as well as validation agents - generally catch these issues. For me, I've had to institute tighter controls and guardrails via hooks but I don't see results that warrant changing to a different provider.
- zmmmmm 6mo agoObviously it's entirely unprovable but it all aligns in very suspicious ways with a compelling narrative: Anthropic simply can't actually scale Claude Code to meet the opportunity right now. Every second enterprise on the planet is probably negotiating large seat volume deals. It's a race for survival against the other players. The sales team is making huge promises engineering and ops can't fulfil. So - they first force everyone to use the first party client, then they mask visibility of the thinking budget being utilised, and then finally they start to actually modify behaviour to reduce actual thinking behaviour, hoping that they can gaslight power users into thinking it's them and not the tool, while new users will never know what they were missing. Is the narrative true? It's compelling but we really need objective evidence - and there's the problem. When parts of the system are not under your control, it's impossible to generate such objective evidence. Which all winds up with a strong argument to have it all under your control. If it didn't happen this time, it probably will. Enshittification is a fundamental human behavioral constant.
- marcyb5st 6mo agoI believe they can't afford anymore to subsidize inference with VC money or that they are trying to get their balance sheet in order for an IPO. So they could be trying to tighten the thinking budget (to decrease tokens per request) or to lobotomize the model (to have cheaper tokens). I mean, no-one is really sure how much a 200 dollars/month plan actually costs Anthropic, but the consensus is "more than that" and that might be coming to an end. This explanation falls well in line with the recent outrage about out of quotas error that people were reporting for the cheaper (or free) plans.
- QuantumNoodle 6mo agoAi tooling is fantastic but not being able to version and control the model into which you pump your dependant workflows is such a liability.
- maxmorrish 6mo agobeen using claude code pretty heavily for the last few months and yeah the context window stuff can be frustrating on bigger codebases. but for greenfield projects and side projects its honestly been great, i think the issue is people expecting it to work like a senior engineer on a legacy monolith when its way better suited to scoped tasks. the trick is breaking things down before you start
- getverdict 6mo ago[dead]
- mahadillah-ai 6mo ago[dead]
- alexchen_dev 6mo ago[flagged]
- another_twist 6mo agoIs it just me that I simply don't care ? I never one-shot these tasks, always provide a breakdown and always give the AI straightforward tasks that would take too much typing. The approach seems to work just fine regardless of the model. If it gets stuck, I usually take over and do the task myself. Also allows me to plan for throughput rather than latency - i.e. start 2-3 small tasks in parallel and do 1 complicated task or planning myself. It works whether I use codex or claude. I lean more towards codex since its cheaper. Even aider gets good results this way.
- rimliu 5mo agoIt's you. Where did you get "one-shot" from that report? One shot or detailed step-by step - claude has gone worse.
- BoorishBears 6mo agoI hope that Anthropic continues to do well and coding agents in general continues to progress... but I also hope Claude Code implodes dramatically and completely so we can get a ground up rebuild with sound engineering. Every week it seems like we're getting closer. Bonus: A high profile case might end people fixating on how long they can go without writing any code. Which makes about as much sense as a mechanic fixating on how long they go between snapped bolts without a torque wrench.
- glasswerkskimny 6mo ago[flagged]
- edinetdb 5mo ago[flagged]
- porridgeraisin 5mo agoIMO, it's an expectations vs reality thing. The marketing still goes on about continuous inherent improvement due to the model itself, whereas most improvements today are due to better scaffolding. The key now is to build tooling around these LLMs to make them reliably productive - whatever level that may be at. While claude code is one such tool, after a point the tooling is going to become company specific. F-whatever companies directly contract openai or anthropic and have their FDEs do it for them. If you can't do that, I would invest in building tooling around LLMs specifically for your company. Note that LLMs are approximate retrieval machines. You still need a planner* and a verifier around it. Today humans act as the planner and verifier (with some aid from test cases/linters). Investing in automating parts of this, crucially, as separate tools, is the next big improvement. * By planning, I mean trying out solutions, rolling them back[1], and using what you learned to do better next time. The solution search process. Context management also falls under this. [1] and no, LLMs going "wait no..." doesn't count.
- rimliu 5mo agoit is past reality vs. current reality. The only expectation here was not to see it degrade that much.
- caiyongji 5mo agoThank you. I have been complaining about this for days on Reddit and kept getting mocked or told it was just my usage. Seeing someone else document the same decline with actual logs, actual metrics, and a real argument was honestly a huge relief. Your issue was posted almost at the same time as my own posts yesterday. That timing hit me hard. Finally, I do not feel like I was shouting into the void.
- wrqvrwvq 5mo agohilarious that there's 10 billion lines of context being shuffled around and argued over but paying a dev 100K is a techno sin. Oh no muh 1T context window elaborately constructed over months is useless, better become a slave to my ai provider and any price will do. plz write my code but for free but for all my company's value.
- wrqvrwvq 5mo agoahhh, i have no idea what i'm doing...! lol , but a bot wrote this, im not even responsible
- wrqvrwvq 5mo agowhat are you morons still even pretending to do??? I agentically replace myself and it sucks not only the entire financial proposition of my company out of itself, but also just sucks. nice ...
- areys 5mo ago[flagged]
- PeterStuer 5mo agoAnnecdotal: I have been battling with Claude Opus on a complex multi step project for nearly 4 days. The initial research plan was sound. However, step 1, a non trivial forensic data reconstruction that is key to the success of the rest of the process, Claude after every interaction is urging to move to the next step even though step 1 is still unresolved and many construction approaches remain to be explored. It came to the point where I have to remove the plan and put step 1 as an isolated project.
- deleted 5mo ago[deleted]
- Havoc 5mo agoTurns out tokens are expensive
- healthy_throw 5mo agoIs the era of succinct bug reports with just a reproducible example attached over? Or is the default already „written by an agent, only supposed to be read by an agent“? Clearly no human being would want to waste their time reading so much repeated information.
- claudexai 5mo ago[dead]
- gib444 5mo agoYou will build nothing and you'll be happy. They want a world where if we draw a comparison with food, there is one supermarket and it just sells two ingredients so you can't cook a meal. McDonald's etc flourish The lie is "supercharged ability to build whatever you want", but the reality soon will be the total opposite Look at how many people have zero cooking skills these days
- ramon156 5mo agoMeh, I had been using Claude Code extensively for a while (since release), and I think the quality has gone to shit. I have no data to back up this claim, so it might be placebo. GLM 5.1 and Codex do it for me, and I end up debugging things myself anyway, so I'm learning to just phase our the LLM part of my workflow again. Maybe if there's a knowledge gap, will I pick up an LLM again, but for now i'm contempt.
- saidnooneever 5mo agothere is a comment on there which feels right, despite it might be too subjective. Ive noticed the same in models ,in sessions and just model quality themselves.. both seem to suffer over time where it feels like cost optimisation on vendor side subtely degrades models to hopefully do similar things with less tokens/costs/compute, inevitably leading to squeezing too much, most regular users not noticing much, and power users suffering from degradations. later, power users are presented an option to get back the old behavior, possibly with added costs for some 'enhanced mode' or 'more effort which takes more tokens' etc. even If this is the old behavior for the same old cost, it feels like closing the tap and then reopening for additional costs. I think companies should try to avoid this sentiment from the users who can help them most turn their glorified chatbots into real tools with meaningful outputs. (ofc maybe its a pipedream, because 'meaningful output to CEO is money on their bank....)
- zkmon 5mo agoIn general, I never allowed Claude to manage the complexity. Claude is fantastic coder, but very bad at higher level work. I engage gemini or qwen top models for anything that happens before getting to write code. Claude gets a very strict and elaborate requirement and design spec that it need to execute without any variation. Claude could get too much creative and bloat it's way for non-coding tasks, as these tasks cannot be "sandboxed" with full specs as it can be done for coding.
- robstertalk 5mo ago[flagged]
- SilverSlash 5mo agoI'm deeply regretting paying for this service right now. There is some gaslighting going on in that issue that it's because of the 1M context model. I am using the non-1M context model and it's still disastrously bad.
- jruz 5mo agoThis is last month I'm on the Max plan is just not worth it anymore, $20 Codex and writing myself to keep my brain functioning is my sweetspot. This people are not your friends, they rot your brain.
- anonyfox 5mo agoYou also divide numbers by hand on paper instead of using a calculator?
- sutterd 5mo agoI still use 4.5. I occasionally try 4.6 but always switch back. The “bias towards action” is what I hate. 4.5 would make sure it understands what I want. 4.6 will just make shit up. Maybe the Anthropic people always write crystal clear instructions so it works for them. For me, I just can’t get 4.6 to do what I want.
- virtualritz 5mo agoMy verdict after last night trying what was suggested here: yes, with CLAUDE_CODE_EFFORT_LEVEL=max (or at least high, for this you don't need to set an env var, it will remember) and CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING=1 you can get Claude to perform as before. I have been using Claude on /effort high since Opus 4.6 rolled out as medium would never get me good enough results (Rust, computer-graphics-related code). I, too, noticed the drop in quality a month or so ago. With CLAUDE_CODE_DISABLE_ADAPTIVE_THINKING=1 it's back to what feels to be pre-March performance -- but then your tokens will 'evaporate' 40% faster. And that was not the case then; I had similar/same performance before but wasn't running out of tokens ever on a Max subscription. So a it's a rug-pull, as before/last late summer, from whatever angle you look at it.
- neuzhou 5mo ago[dead]
- k9294 5mo agoAnecdotally, I’ve been seeing a lot of weird behavior from Opus when it decides, mid-execution, to switch to a different "simpler" solution, and that really pissed me off. At one point, I carefully designed a spec document, forced Opus to reread it, create a plan with the planning tool that followed the spec, and use the task tool to track the implementation... AND AFTER OPUS READS THE FIRST FUCKING FILE, it says, "Oh, there are missing dependencies in project X. It’ll be hard to add them, so I’m going to throw away the whole plan and just do a simple fix..." After that, I canceled my $200 Max plan, which I’d been subscribed to since June 2025, and decided to check out Codex
- mt18 5mo agoFebruary is a red herring—most teams never wrote down what human-owned correctness means once the model touches prod.
- tontinton 5mo agoI don't know why everyone is so attached to Claude Code you can just build your own little agent, like I did: https://maki.sh/ https://maki.sh/ It will 100% be better than the 500k lines of code junk that is CC.
- BrianFHearn 5mo ago[dead]
- torrienaylor 5mo agoOur team has been using claude extensively and has started to hit some of the same context wall. Specifically around knowledge transfer. Everyone is coding, including PMs and Design and the classic 1x1 and team stand-up rotation wasn't keeping up. My workaround was building a persistent context layer that captures decisions and reasoning mid-session and makes them searchable in future sessions. Consider this a "Team Memory".
- giant-broccoli 5mo ago[dead]
- traderaegis 5mo ago[dead]
- noisy_boy 5mo agoWhen I have 10-20 minutes to spare about doing sort of a one-shot-thoughtful change, I go for Claude Code. Problem is that a) I'm waiting for quite a while and b) My waiting isn't always fruitful because it does get things wrong in which case I correct it and off it goes for another 5-10 minute expedition. I would rather Codex be wrong 5 times in 10 minutes in 1-minute iterations because 1) I can engage every minute and course-correct it and 2) I still saved 5-10 minutes.
- sillyboi 5mo ago[dead]
- LetsGetTechnicl 5mo agoPeople finding out in real time that LLM's are not economically viable and this is one way AI companies are trying to squeeze any amount of profit out of it, by making it worse. Happened before, AI is just that unprofitable
- ssz7820 5mo ago[dead]
- data-ottawa 5mo agoI reviewed 118 conversations with Claude since March 6, all on real work projects. Each conversation was processed to assess level of frustration, source of frustration, and evaluated with Gemma 4 and Claude Opus for spot checking. I have a tool I use to manage my work trees, so most work has is done on branches prefixed with ad-hoc/feature/explore or similar, and data was tagged with branch names. 43% of my Claude Code sessions (Opus 4.6, high reasoning) ended with signals of frustration. 73% of total chat time (by total messages) was spent in conversations which were eventually ranked as frustrating. Median time to frustration was 25 messages, and on average, each message from Claude has about a baseline 5% chance of being frustrating. Frustration by chat length actually matches this 5% baseline of IID Bernoullis -- which is surprising and interesting, as this should not be IID at all. Frustration types: - Wrong answers – 14% of sessions, 31% of frustration - Instruction Following – 11% of sessions, 25% of frustration - Overcomplication – 8% of sessions, 18% of frustration - Destructive Actions (e.g. requesting to delete something or commit a change to prod) – 3% of sessions, 8% of frustration - Non-responsive (service outages leading to non-response) 2% of sessions - Miscommunication 2% of sessions - Failed execution 2% of sessions Half of frustrations happened in the first or last 20% of a chat by length. I interpret early frustrations to be recoverable, late frustrations to be terminal. Early frustrations (sessions averaged 45 turns): - 30% overcomplicating the problem - 30% instruction following issues - 30% wrong answers - 10% destructive actions Late frustrations (sessions averaged 12 turns -- i.e. terminal context early) - 36% Wrong answers, with repetition - 21% instruction following, with repeated correction from user (me) - 14% Service interruptions/outages - 7% failed execution - 7% communication - Claude is unable to articulate some result, or understand the problem correctly. Late frustrations led to the highest levels of frustration, 29% of the time. I'm a data scientist — my most frustrating work with Claude was data cleaning/repair (a complex backfill) issues -- with 75% of sessions marked frustrating due to overcomplicating, instruction following, or destructive actions). The best (least frustrating) workflows for DS were code-review, scoped feature work (with tickets), data validation, and config/setup tasks and automation. Ad-hoc query work ended up in between -- ad-hoc requests were generally bootstrapping queries or doing rough analysis on good data. Side note: all of my interactions with the /buddy feature were flagged as high frustration ("furious"). That was a false positive over mock arguing with it, but did provide a neat calibration signal. Those sessions were removed entirely from the analysis after classification.
- tanseydavid 5mo agoThank you for making this detailed analysis and write up.
- namirsab 5mo agoThis seems to keep happening, I just had a situation were Claude told me to try a thing, I tried, it didn't work, then It told me to try another thing, I tried, it didn't work, and finally it asked me again to try the first thing again, as if we never tested it before, even though it was in the context from 3 messages ago. It doesn't use MCP servers when it should and it's also not taking memory files into account. This is happening with /effort high and in really simple tasks... :(
- lyncyan 5mo ago[flagged]
- tomaskafka 5mo agoI am not happy with the fact that 2026 was the last year a normie without $10M+ enterprise account could access SOTA models in a non-demo mode.
- glasswerkskimny 5mo ago[dead]
- cordwainersmith 5mo agoWe rolled it out across ~1k engineers and the biggest issue wasn't the model quality, it was observability. Nobody could tell me if the agent was stuck in a loop, which sessions were expensive, or what the cache hit rate looked like. Without that visibility you can't distinguish "the model is bad" from "my setup is bad." Most of the complaints we got early on turned out to be config problems.
- bicepjai 5mo ago[dead]
- roninforge 5mo ago[dead]
- liamsfr 5mo agoSo: 1/ lack of thinking in transcripts is not a decisive metric for determining if any thinking was done, but 2/ the reply does not address the qualitative aspects that Stella’s team observed and provided data for from what amounted to a bad qualitative experience with serious financial implications. It’s a sidestep for explaining away the research, but does not address the underlying issue: has quality been degrading (selectively, intentionally or otherwise)?
- drpython 5mo agoMoved to Codex and breathing fresh air.
- YaraDori 5mo ago[dead]
- ai-tamer 5mo ago[dead]
- pedriellis 5mo ago[dead]
- macformula2gx 5mo agoClaude is great I really love it but keen to know will you ever have one plan and one history / context across all the different tools you have ? agents on cloud - platform console, cli, desktop(chat+code+cowork), claude.ai, chrome addons ? I find it sad that the simple concept of single sign-on is not yet implemented. The history of conversations is not across all conversations. the choice of model switch in between or automatically is not there. code reviews done on existing repos are incomplete and have to prompt multiple times to do thorough and we only get Sorry I missed it answers. Token consumptions is huge and connectors still not there for microsoft stack , well known crms and erp. and you need to oay seperatrlt for api versus platform versus others. there - I said it ;) Hoping the next version will be truly for developers .