6 ms·

I’ve definitely experienced step jumps down in quality on an almost daily basis. I usually used xhigh. The experience of relying on codex’s outstandingly thorou

I’ve definitely experienced step jumps down in quality on an almost daily basis. I usually used xhigh. The experience of relying on codex’s outstandingly thorough coding earlier in the year has evaporated for me. I’m seeing incredibly stupid implementations intermittently, and have simply switched to Claude until openai takes the issue seriously. As far as i could tell they haven’t taken it seriously for the several months I’ve been personally seeing it.

Losenok 2mo ago
[dead]
siva7 2mo ago
I've switched 3 months ago to Codex because Claude got incredibly stupid. 6 months ago vice versa. It doesn't matter if you use Codex or Claude. Both will fuck with you at some point. Though Codex probably less.
- selectodude 2mo ago
  At least OpenAI lets me use my own harness. Having to rely on insane PMs letting Claude Mythos go wild on the codebase has not been going well lately.
  - tunesmith 2mo ago
    How do you mean OpenAI lets you use your own harness? I'm under the impression that a custom harness requires the OpenAI SDK, which requires api tokens rather than plus/pro accounts.
    - HumanOstrich 2mo ago
      Anthropic is the one that prohibits harnesses other than Claude Code on subscription plans and bans users for disobeying. OpenAI officially allows that with subscriptions.
    - hakunin 2mo ago
      You must've missed the OpenAI's response to Anthropic forcing everyone to their own harness if they want subscription pricing: official endorsement of custom harnesses like opencode and pi even when used with Codex subscription. I think they even partnered with opencode or something like that (don't remember).
    - lhl 2mo ago
      https://x.com/thsottiaux/status/2058071172361998482 https://x.com/thsottiaux/status/2058071172361998482 "A little secret. About 5% of our production traffic is on the Pi harness, about another 5% is on OpenCode. Reminder you can use your ChatGPT account in a flourishing set of other tools. We’ll continue to make Codex awesome, but you have options."
      - deleted 2mo ago
        [deleted]
      - solenoid0937 2mo ago
        They will only do this while they have no users. Once they are compute constrained they won't let you do this anymore
        selectodude 2mo ago
        That’s the beauty of using an open harness. I can jump ship if that ever happens.
    - smoe 2mo ago
      Not just harnesses, you can even use the subscription in CI/CD. That, plus the fact that web chat does not count toward the same limits, is why I think the Codex personal plan is easily 10x the value of Claude Code. https://developers.openai.com/codex/auth/ci-cd-auth https://developers.openai.com/codex/auth/ci-cd-auth
      - 7thpower 2mo ago
        How are you using it in ci/cd?
      - gck1 2mo ago
        I'm never going back to claude from codex, including for the reasons you mentioned, but it must be said that web chat inference on ChatGPT is magic incantation, and I'm almost certain they're not serving the same models there as in Codex. Claude web definitely feels like it's the same models behind as in API, with much less extra behavior/layers that make it behave differently.
    - miki123211 2mo ago
      OpenAI doesn't require that; only Anthropic does. OpenAI's harness is fully open source[1], and (AFAIK) doesn't come with any kind of signed-build request integrity verification like Claude does. And by that logic, if you're allowed to use their API with a fork of Codex that you yourself compiled, there's nothing stopping you from making some other harness act like such a fork. [1] https://github.com/openai/codex https://github.com/openai/codex
      - lmwnshn 2mo ago
        And although Codex Desktop is not open source (AFAIK), it does expose CDP and appserver which allowed me to build a Greasemonkey-like plugin system on top of it. It's surprisingly powerful, you can add transcript annotations, programmatically control the side panels, integrate a native-looking account switcher, and so on.
        walthamstow 2mo ago
        This sounds really cool. Do you have the source published anywhere?
        lmwnshn 2mo ago
        Yup! MIT licensed, do whatever you want, you can also just use it directly with nodejs installed through `npx clankerbend` [0]. The screenshot on the github repo links you to a YouTube demo. Warning: I use this daily myself for work, but beyond me arguing with it to get the CDP architecture, this was completely vibe-coded. I mostly use vim mode (jump to line, jump to prev/next user message) and the account switcher. [0] https://github.com/onewillai/clankerbend https://github.com/onewillai/clankerbend
  - navigate8310 2mo ago
    Will this problem not arise on other open source harness?
- jerezzprime 2mo ago
  If you use GitHub Copilot you can switch between them in the same session if you want.
  - gwerbin 2mo ago
    Yeah but now that you pay for tokens that's going to be bad for token caching.
  - ElFitz 2mo ago
    Same with a third-party open harness.
cyanydeez 2mo ago
i don't ever believe these issues are technical. They're business decisions to downgrade performance because to fix it means $$$$ and you arn't paying them enough.
matco11 2mo ago
I have noticed this degradation of 5.5 reliability to what, in my experience, I consider Claude-level of reliability since early June. My journey dealing with this has been transitioning from 5.5 high to 5.5 xhigh to 5.4 high. 5.4 high has been perfectly reliable for me for the last 3 weeks, and I am happy there. Occasionally, I run some tasks on 5.5 xhigh to check if it has gone back to being 100% perfectly reliable, but, at this point, I am assuming they are just counting on releasing 5.6 rather than dealing with this reliability issue.
- beacon294 2mo ago
  I'm on the same journey but I bought a 3090 and put qwen 3.6 27b on it. It covers some things with better reliability. Obviously it doesn't have the breadth of a large model. If that's even a selling point for large models for coding?

Losenok 2mo ago

[dead]

siva7 2mo ago

I've switched 3 months ago to Codex because Claude got incredibly stupid. 6 months ago vice versa. It doesn't matter if you use Codex or Claude. Both will fuck with you at some point. Though Codex probably less.

selectodude 2mo ago
At least OpenAI lets me use my own harness. Having to rely on insane PMs letting Claude Mythos go wild on the codebase has not been going well lately.
- tunesmith 2mo ago
  How do you mean OpenAI lets you use your own harness? I'm under the impression that a custom harness requires the OpenAI SDK, which requires api tokens rather than plus/pro accounts.
  - HumanOstrich 2mo ago
    Anthropic is the one that prohibits harnesses other than Claude Code on subscription plans and bans users for disobeying. OpenAI officially allows that with subscriptions.
  - hakunin 2mo ago
    You must've missed the OpenAI's response to Anthropic forcing everyone to their own harness if they want subscription pricing: official endorsement of custom harnesses like opencode and pi even when used with Codex subscription. I think they even partnered with opencode or something like that (don't remember).
  - lhl 2mo ago
    https://x.com/thsottiaux/status/2058071172361998482 https://x.com/thsottiaux/status/2058071172361998482 "A little secret. About 5% of our production traffic is on the Pi harness, about another 5% is on OpenCode. Reminder you can use your ChatGPT account in a flourishing set of other tools. We’ll continue to make Codex awesome, but you have options."
    - deleted 2mo ago
      [deleted]
    - solenoid0937 2mo ago
      They will only do this while they have no users. Once they are compute constrained they won't let you do this anymore
      - selectodude 2mo ago
        That’s the beauty of using an open harness. I can jump ship if that ever happens.
  - smoe 2mo ago
    Not just harnesses, you can even use the subscription in CI/CD. That, plus the fact that web chat does not count toward the same limits, is why I think the Codex personal plan is easily 10x the value of Claude Code. https://developers.openai.com/codex/auth/ci-cd-auth https://developers.openai.com/codex/auth/ci-cd-auth
    - 7thpower 2mo ago
      How are you using it in ci/cd?
    - gck1 2mo ago
      I'm never going back to claude from codex, including for the reasons you mentioned, but it must be said that web chat inference on ChatGPT is magic incantation, and I'm almost certain they're not serving the same models there as in Codex. Claude web definitely feels like it's the same models behind as in API, with much less extra behavior/layers that make it behave differently.
  - miki123211 2mo ago
    OpenAI doesn't require that; only Anthropic does. OpenAI's harness is fully open source[1], and (AFAIK) doesn't come with any kind of signed-build request integrity verification like Claude does. And by that logic, if you're allowed to use their API with a fork of Codex that you yourself compiled, there's nothing stopping you from making some other harness act like such a fork. [1] https://github.com/openai/codex https://github.com/openai/codex
    - lmwnshn 2mo ago
      And although Codex Desktop is not open source (AFAIK), it does expose CDP and appserver which allowed me to build a Greasemonkey-like plugin system on top of it. It's surprisingly powerful, you can add transcript annotations, programmatically control the side panels, integrate a native-looking account switcher, and so on.
      - walthamstow 2mo ago
        This sounds really cool. Do you have the source published anywhere?
        lmwnshn 2mo ago
        Yup! MIT licensed, do whatever you want, you can also just use it directly with nodejs installed through `npx clankerbend` [0]. The screenshot on the github repo links you to a YouTube demo. Warning: I use this daily myself for work, but beyond me arguing with it to get the CDP architecture, this was completely vibe-coded. I mostly use vim mode (jump to line, jump to prev/next user message) and the account switcher. [0] https://github.com/onewillai/clankerbend https://github.com/onewillai/clankerbend
- navigate8310 2mo ago
  Will this problem not arise on other open source harness?
jerezzprime 2mo ago
If you use GitHub Copilot you can switch between them in the same session if you want.
- gwerbin 2mo ago
  Yeah but now that you pay for tokens that's going to be bad for token caching.
- ElFitz 2mo ago
  Same with a third-party open harness.

cyanydeez 2mo ago

i don't ever believe these issues are technical. They're business decisions to downgrade performance because to fix it means $$$$ and you arn't paying them enough.

matco11 2mo ago

I have noticed this degradation of 5.5 reliability to what, in my experience, I consider Claude-level of reliability since early June. My journey dealing with this has been transitioning from 5.5 high to 5.5 xhigh to 5.4 high. 5.4 high has been perfectly reliable for me for the last 3 weeks, and I am happy there. Occasionally, I run some tasks on 5.5 xhigh to check if it has gone back to being 100% perfectly reliable, but, at this point, I am assuming they are just counting on releasing 5.6 rather than dealing with this reliability issue.

beacon294 2mo ago
I'm on the same journey but I bought a 3090 and put qwen 3.6 27b on it. It covers some things with better reliability. Obviously it doesn't have the breadth of a large model. If that's even a selling point for large models for coding?