5 ms·

Claude Code injects a ton of tools into the system prompt, including their “memory system” that’s like 10k+ tokens. Depending on your task shape, this can easil

by yojo 2mo ago

Claude Code injects a ton of tools into the system prompt, including their “memory system” that’s like 10k+ tokens. Depending on your task shape, this can easily double your task cost (e.g. a low context-using job that takes many turns, like a monitoring loop). You should use --disallowed-tools to prune any tools not needed for the task. Note that this is also a perpetual game of whack-a-mole since they’re always adding new tools.

visarga 2mo ago
The system prompt part is surely cached across all users.
- dcrazy 2mo ago
  For Anthropic models, yes. But OP was using Claude Code with GPT.
- yojo 2mo ago
  You still pay cache token costs on API calls. Cache cost/token are 90% lower, but you pay it every single turn. I’m not sure if they let you skip the cache write cost on the first turn. That would imply cross-user caching infrastructure or special casing the default system prompt to give you a discount. Maybe? Away from the computer but you could try a “hello” in a fresh session and see what was billed.
  - cyanydeez 2mo ago
    Its inexpensive to reserve KV cache for the first turn and would benefit users, given the first turn already requires costly locating and allocating a model slot for a user. So yeah, when they banned tgird party harnesses there was a technical and $ case to have.
- tosh 2mo ago
  the system prompt still takes up useful space in the context window and steers the model into unnecessary actions and over-thinking patterns
tosh 2mo ago
ty re --disallowed-tools for coding agents 'shell' is often all you need (just make sure the environment has the necessary tools)
KronisLV 2mo ago
> Claude Code injects a ton of tools into the system prompt It's so unfortunate they don't let you use the subscription with other harnesses anymore - since even if I used OpenCode they'd still get a bunch of useful data from the API calls, meanwhile I could stretch their tier limits way further.
- epolanski 2mo ago
  It's so unfortunate people don't realize it's cheaper to write their own well working agent instead of insisting with general purpose bloated ones like Claude. 200$/month is a lot of money on Luna/DS4 flash, like really a lot and the results are much better than clowning on bloated CC. It's absurd how you have more and more organizations encoding their processes on LLMs and "engineers" (charlatan coders) don't even bother optimizing the tool they use most.
  - RussianCow 2mo ago
    > the results are much better than clowning on bloated CC. I won't argue with the cost effectiveness, but the results are very much not better. Opus and Fable are in a different league than DS4 Flash. Even GPT Terra, which I really like overall, sometimes gets stuck in weird loops and starts to do stupid stuff once its context window fills up. Whereas I can more or less trust the big models to just Do The Thing™ on the first try. With that said, you get way more value out of a GPT subscription than you do from Claude, partly because of the ability to use more efficient harnesses.
    - epolanski 2mo ago
      You're confusing models with the agents. You can use opus or fable if you please, I'm advocating for writing your own agent instead of used generic bloated ones like CC.
      - RussianCow 2mo ago
        I'm not confusing anything since you can't use custom harnesses with your Claude subscription—you have to use Claude Code. So as far as the Claude sub is concerned, the models and the harness are coupled.
colourclash 2mo ago
appreciate the tip, kind netizen

by yojo 2mo ago

visarga 2mo ago

The system prompt part is surely cached across all users.

dcrazy 2mo ago
For Anthropic models, yes. But OP was using Claude Code with GPT.
yojo 2mo ago
You still pay cache token costs on API calls. Cache cost/token are 90% lower, but you pay it every single turn. I’m not sure if they let you skip the cache write cost on the first turn. That would imply cross-user caching infrastructure or special casing the default system prompt to give you a discount. Maybe? Away from the computer but you could try a “hello” in a fresh session and see what was billed.
- cyanydeez 2mo ago
  Its inexpensive to reserve KV cache for the first turn and would benefit users, given the first turn already requires costly locating and allocating a model slot for a user. So yeah, when they banned tgird party harnesses there was a technical and $ case to have.
tosh 2mo ago
the system prompt still takes up useful space in the context window and steers the model into unnecessary actions and over-thinking patterns

tosh 2mo ago

ty re --disallowed-tools for coding agents 'shell' is often all you need (just make sure the environment has the necessary tools)

KronisLV 2mo ago

> Claude Code injects a ton of tools into the system prompt It's so unfortunate they don't let you use the subscription with other harnesses anymore - since even if I used OpenCode they'd still get a bunch of useful data from the API calls, meanwhile I could stretch their tier limits way further.

epolanski 2mo ago
It's so unfortunate people don't realize it's cheaper to write their own well working agent instead of insisting with general purpose bloated ones like Claude. 200$/month is a lot of money on Luna/DS4 flash, like really a lot and the results are much better than clowning on bloated CC. It's absurd how you have more and more organizations encoding their processes on LLMs and "engineers" (charlatan coders) don't even bother optimizing the tool they use most.
- RussianCow 2mo ago
  > the results are much better than clowning on bloated CC. I won't argue with the cost effectiveness, but the results are very much not better. Opus and Fable are in a different league than DS4 Flash. Even GPT Terra, which I really like overall, sometimes gets stuck in weird loops and starts to do stupid stuff once its context window fills up. Whereas I can more or less trust the big models to just Do The Thing™ on the first try. With that said, you get way more value out of a GPT subscription than you do from Claude, partly because of the ability to use more efficient harnesses.
  - epolanski 2mo ago
    You're confusing models with the agents. You can use opus or fable if you please, I'm advocating for writing your own agent instead of used generic bloated ones like CC.
    - RussianCow 2mo ago
      I'm not confusing anything since you can't use custom harnesses with your Claude subscription—you have to use Claude Code. So as far as the Claude sub is concerned, the models and the harness are coupled.

colourclash 2mo ago

appreciate the tip, kind netizen