5 ms·

This post and comment makes me believe "science" is the new "code" for Anthropic now that the code advantage is mostly gone and lost for OpenAI, ie. they got mu

This post and comment makes me believe "science" is the new "code" for Anthropic now that the code advantage is mostly gone and lost for OpenAI, ie. they got much better and Claude become significantly worse over these months.

echelon 15d ago
IMO, Codex is worse than Claude with Fable. At least at Rust. That said, the open source models are not bad and I'm looking forward to more tools and products built on top of them. Code review, security review, etc. Anthropic needs to change how it treats users though. I'm increasingly put off by Dario, the rug pulling, the lies, and the attempts to regulate open weights. I'm going to bail if this doesn't change. There's plenty enough that's good enough, and those things are hackable and extensible. If Fable isn't available at subscription price via third party harnesses soon, I'm also going to bail.
- ed-is-ai 15d ago
  The big issue I have with Fable is this. From the Anthropic email announcing Fable 5.1. So basically they're giving us a Ferrari, which will point blank refuse to do certain stuff - forcing us to go out in our Mustang. Their choice, not ours "Safeguards and automatic fallbacks (beta): Fable 5.1’s biology and cybersecurity classifiers block fewer benign requests and now permit vulnerability finding in source code. Blocked requests return an error and are not charged to you. On the Messages API, opt in to fall back to another model so users get a response instead of an error. We recommend Opus 5 for biology and Opus 4.8 for cybersecurity. In Managed Agents, fallback is built in."
- ImprobableTruth 15d ago
  Its "pure capabilities" are definitely worse than Fable, but I find codex has a much more pleasant style and is in comparison much more generous with its limits.
  - dmix 15d ago
    Codex (+Sol) feels a lot more human for sure. Fable 5 is so, so wordy.
  - TuxSH 15d ago
    It used to be true up to 2w ago, but with the new/reinstated 5h limits I wouldn't be so sure anymore...
    - panos_news 15d ago
      Claude has a better 5hr limit?
    - selectodude 15d ago
      that's news to me, I'm still getting weekly limits, no hourly limits.
  - versteegen 15d ago
    Ugh, people are still saying the Codex limits are more generous. They're not, Claude's are over 2x higher, have been for months! [1] It's just that Claude uses far more tokens, 2-3x is common. Except sometimes GPT will use just as many or even go into a compact loop and then your quota is gone, little headroom for hard tasks. [1] https://devforth.io/agents-for-code/?sortby=monthly-value https://devforth.io/agents-for-code/?sortby=monthly-value And I can confirm the numbers, I subscribe to both and watch the numbers
    - stevego7 15d ago
      Yeah I subscribe to both and watch the numbers too, and it drives me nuts Who wants to actually watch anyways, rather than worry about it my team just created our own harness that prioritize usage + intelligence and assigns work out (and records token usage..) https://go7workhorse.com https://go7workhorse.com Still beta, please try it and give me feedback.
    - trentor 15d ago
      I don't get it. It's the same result.
- re-thc 15d ago
  > IMO, Codex is worse than Claude with Fable. Fable easily trips its safe guards. You can be 95% complete with the plan for it to trip and then lose it all. Anything is better than nothing.
  - enraged_camel 15d ago
    >> Fable easily trips its safe guards. Maybe it depends on the type of work you do, because for me it almost never happens. >> You can be 95% complete with the plan for it to trip and then lose it all. That's... not what happens though. The session will either seamlessly downgrade to another model mid-session, or it will stop with an alert and you can just re-prompt it. It will still have access to the context.
    - ipsod 15d ago
      Web apps are where I have this trouble. Making a web app secure is literally just finding and patching vulnerabilities, instead of finding and exploiting them. You could have the AI "try to make this app secure", find what it patches, and use it for exploits, and the AI can't know if that's what you're trying to do or not. I don't know how you can get around this. I get around it by not using Anthropic products, at present.
      - Exoristos 15d ago
        Not to endorse OpenAI's particular guardrails, but unless you're doing something groundbreaking, security best practices should be more than enough for web development.
        re-thc 15d ago
        With OpenAI you can also apply for the security program, which doesn't require you to be a certified pentester (as per Anthropic).
        ipsod 15d ago
        OpenAI is what I use most. Sol 5.6 still rejects a few requests a day when I'm working on web apps, but, overall, it's not too bad. I wish it'd auto-resume and try again, instead of waiting for me to intervene, but it's rare enough that it's not a huge deal. It probably doesn't help that I'm using frameworkless PHP - I imagine a lot triggers could be avoided if I was using a framework where secure features were baked in.
- boc 15d ago
  Small reminder that the US government rug-pulled Fable, not Dario. Lots of the safety guards that users find annoying/objectionable were the results of negotiations to get the model back online after the US government forced them to take it down. Maybe Dario should have just "donated" $1M to Trump's inauguration fund like Altman, Meta, Amazon, Microsoft, Tim Cook, Elon, and Google. There's a reason they are the odd man out with this current Administration.
  - rudedogg 15d ago
    Maybe Dario shouldn’t have tried for regulatory capture. He was constantly on the news talking about how these models are so dangerous and that we need regulation to keep China from releasing open-source models without guardrails.
  - sidrag22 15d ago
    The US government didn't make the choices to release the worst version of Opus and label it 5.0, and then isolate portions of their subscribers to limited usage of Fable. They may have been unfairly targeted by the US government, but they are doing more damage to themselves without government help as well.
    - usef- 15d ago
      Fable only being temporarily included in cheaper subscriptions was because anthropic is severely GPU constrained. They still are, and it impacts almost all of those unpopular decisions. They did announce from the beginning it was temporary.
      - sidrag22 15d ago
        Horrifying excuse, gpu constraint can be used by all of these companies to justify a shit user experience. If the user isn't properly weighed in their priorities, they have their priorities setup wrong. Their 20$ tier currently isn't serving their best model, and they insulted their users by putting out an ill tested opus 5.0, which is the worst experience ive personally had using a model in probably 2 years(obviously adjusting for expectations at the time of release).
rot256 15d ago
I write a lot of Rust and Lean, Fable 5 is in my experience better at both. Cost/performance is a different story.
- rowanG077 15d ago
  This is really it imo. Fable 5 is better then Sol. But Fable is just of the table for anything even remotely long running. Unless you have very deep pockets. And the difference between Fable and Sol is not world shattering if you ask me. I also find codex a ton better than claude.
  - prox 15d ago
    So when do you use Fable? For difficult singular tasks?
    - rowanG077 15d ago
      Yes, I use it when Sol Ultra fails to find a solution.
- airstrike 15d ago
  Yes, both of which are domains for which a verifier is readily available. You can generalize from them to "science".
- black_knight 15d ago
  Fable has become my go to in Agda as well. It just crunches hard technical tasks! I find Fable 5 still lacking in library design. But I guess there is no accounting for taste…

echelon 15d ago

IMO, Codex is worse than Claude with Fable. At least at Rust. That said, the open source models are not bad and I'm looking forward to more tools and products built on top of them. Code review, security review, etc. Anthropic needs to change how it treats users though. I'm increasingly put off by Dario, the rug pulling, the lies, and the attempts to regulate open weights. I'm going to bail if this doesn't change. There's plenty enough that's good enough, and those things are hackable and extensible. If Fable isn't available at subscription price via third party harnesses soon, I'm also going to bail.

ed-is-ai 15d ago
The big issue I have with Fable is this. From the Anthropic email announcing Fable 5.1. So basically they're giving us a Ferrari, which will point blank refuse to do certain stuff - forcing us to go out in our Mustang. Their choice, not ours "Safeguards and automatic fallbacks (beta): Fable 5.1’s biology and cybersecurity classifiers block fewer benign requests and now permit vulnerability finding in source code. Blocked requests return an error and are not charged to you. On the Messages API, opt in to fall back to another model so users get a response instead of an error. We recommend Opus 5 for biology and Opus 4.8 for cybersecurity. In Managed Agents, fallback is built in."
ImprobableTruth 15d ago
Its "pure capabilities" are definitely worse than Fable, but I find codex has a much more pleasant style and is in comparison much more generous with its limits.
- dmix 15d ago
  Codex (+Sol) feels a lot more human for sure. Fable 5 is so, so wordy.
- TuxSH 15d ago
  It used to be true up to 2w ago, but with the new/reinstated 5h limits I wouldn't be so sure anymore...
  - panos_news 15d ago
    Claude has a better 5hr limit?
  - selectodude 15d ago
    that's news to me, I'm still getting weekly limits, no hourly limits.
- versteegen 15d ago
  Ugh, people are still saying the Codex limits are more generous. They're not, Claude's are over 2x higher, have been for months! [1] It's just that Claude uses far more tokens, 2-3x is common. Except sometimes GPT will use just as many or even go into a compact loop and then your quota is gone, little headroom for hard tasks. [1] https://devforth.io/agents-for-code/?sortby=monthly-value https://devforth.io/agents-for-code/?sortby=monthly-value And I can confirm the numbers, I subscribe to both and watch the numbers
  - stevego7 15d ago
    Yeah I subscribe to both and watch the numbers too, and it drives me nuts Who wants to actually watch anyways, rather than worry about it my team just created our own harness that prioritize usage + intelligence and assigns work out (and records token usage..) https://go7workhorse.com https://go7workhorse.com Still beta, please try it and give me feedback.
  - trentor 15d ago
    I don't get it. It's the same result.
re-thc 15d ago
> IMO, Codex is worse than Claude with Fable. Fable easily trips its safe guards. You can be 95% complete with the plan for it to trip and then lose it all. Anything is better than nothing.
- enraged_camel 15d ago
  >> Fable easily trips its safe guards. Maybe it depends on the type of work you do, because for me it almost never happens. >> You can be 95% complete with the plan for it to trip and then lose it all. That's... not what happens though. The session will either seamlessly downgrade to another model mid-session, or it will stop with an alert and you can just re-prompt it. It will still have access to the context.
  - ipsod 15d ago
    Web apps are where I have this trouble. Making a web app secure is literally just finding and patching vulnerabilities, instead of finding and exploiting them. You could have the AI "try to make this app secure", find what it patches, and use it for exploits, and the AI can't know if that's what you're trying to do or not. I don't know how you can get around this. I get around it by not using Anthropic products, at present.
    - Exoristos 15d ago
      Not to endorse OpenAI's particular guardrails, but unless you're doing something groundbreaking, security best practices should be more than enough for web development.
      - re-thc 15d ago
        With OpenAI you can also apply for the security program, which doesn't require you to be a certified pentester (as per Anthropic).
      - ipsod 15d ago
        OpenAI is what I use most. Sol 5.6 still rejects a few requests a day when I'm working on web apps, but, overall, it's not too bad. I wish it'd auto-resume and try again, instead of waiting for me to intervene, but it's rare enough that it's not a huge deal. It probably doesn't help that I'm using frameworkless PHP - I imagine a lot triggers could be avoided if I was using a framework where secure features were baked in.
boc 15d ago
Small reminder that the US government rug-pulled Fable, not Dario. Lots of the safety guards that users find annoying/objectionable were the results of negotiations to get the model back online after the US government forced them to take it down. Maybe Dario should have just "donated" $1M to Trump's inauguration fund like Altman, Meta, Amazon, Microsoft, Tim Cook, Elon, and Google. There's a reason they are the odd man out with this current Administration.
- rudedogg 15d ago
  Maybe Dario shouldn’t have tried for regulatory capture. He was constantly on the news talking about how these models are so dangerous and that we need regulation to keep China from releasing open-source models without guardrails.
- sidrag22 15d ago
  The US government didn't make the choices to release the worst version of Opus and label it 5.0, and then isolate portions of their subscribers to limited usage of Fable. They may have been unfairly targeted by the US government, but they are doing more damage to themselves without government help as well.
  - usef- 15d ago
    Fable only being temporarily included in cheaper subscriptions was because anthropic is severely GPU constrained. They still are, and it impacts almost all of those unpopular decisions. They did announce from the beginning it was temporary.
    - sidrag22 15d ago
      Horrifying excuse, gpu constraint can be used by all of these companies to justify a shit user experience. If the user isn't properly weighed in their priorities, they have their priorities setup wrong. Their 20$ tier currently isn't serving their best model, and they insulted their users by putting out an ill tested opus 5.0, which is the worst experience ive personally had using a model in probably 2 years(obviously adjusting for expectations at the time of release).

rot256 15d ago

I write a lot of Rust and Lean, Fable 5 is in my experience better at both. Cost/performance is a different story.

rowanG077 15d ago
This is really it imo. Fable 5 is better then Sol. But Fable is just of the table for anything even remotely long running. Unless you have very deep pockets. And the difference between Fable and Sol is not world shattering if you ask me. I also find codex a ton better than claude.
- prox 15d ago
  So when do you use Fable? For difficult singular tasks?
  - rowanG077 15d ago
    Yes, I use it when Sol Ultra fails to find a solution.
airstrike 15d ago
Yes, both of which are domains for which a verifier is readily available. You can generalize from them to "science".
black_knight 15d ago
Fable has become my go to in Agda as well. It just crunches hard technical tasks! I find Fable 5 still lacking in library design. But I guess there is no accounting for taste…