5 ms·

Every single article and banchmark say pi saves token by default, the more I add extensions the more token hungry it gets. pi used 2-3x the tokens of codex. pi

Every single article and banchmark say pi saves token by default, the more I add extensions the more token hungry it gets. pi used 2-3x the tokens of codex. pi with subagent pkg used 8x-10x the tokens of codex. I don't see how adding bloat to pi would make it more token efficient if the baseline is so poor to start with

kadoban 8d ago
You'd have to look where the extra tokens are coming from. If it's using extra turns or doing extra work because a tool it ~wants is missing, then adding extra things will help. Otherwise, it won't. If it's missing guidance that would help, system prompt additions might help. 8x-10x the tokens is wild, is this for some tiny artificial benchmark? That's just too much extra for something not to be just broken.
- toasty228 8d ago
  It was a real workload from my day job, a modest merge request, I asked for a review and for an implementation plan to fix the findings, nothing crazy. It's the most simple thing I could think of that wasn't an artificial benchmark, self contained, no need to look for extra documentation, web searches, etc.
  - kadoban 7d ago
    Yeah something must be actively broken or just an awful extension for it to have that much effect. Most things you can mess with the big effect is like, oh a thousand tokens ended up in the ~system prompt, or 10% extra or fewer work based on extra tool calls or churning through thinking or whatever. Harness stuff if it's 8x worse that's like, it's fucked and broken, something went _wrong_.
crossroadsguy 8d ago
I don't know what you tried but I tried to do something in Pi what I'd have usually done in Claude and it devoured tokens (much more than Claude) and I wasn't even close to finishing the task. Mostly because the barebones Pi was/is very inefficient at anything with any weight. Then I began to customise it to be as good as Claude but eat less taken. I got tired and I had not even scratched the surface. Gave up. I finally realised, at least for me, Pi's best use case is - strip even the little "extra" Pi comes/starts with and then use it just like that if you have a task/work that is appropriate for that bareness.

kadoban 8d ago

You'd have to look where the extra tokens are coming from. If it's using extra turns or doing extra work because a tool it ~wants is missing, then adding extra things will help. Otherwise, it won't. If it's missing guidance that would help, system prompt additions might help. 8x-10x the tokens is wild, is this for some tiny artificial benchmark? That's just too much extra for something not to be just broken.

toasty228 8d ago
It was a real workload from my day job, a modest merge request, I asked for a review and for an implementation plan to fix the findings, nothing crazy. It's the most simple thing I could think of that wasn't an artificial benchmark, self contained, no need to look for extra documentation, web searches, etc.
- kadoban 7d ago
  Yeah something must be actively broken or just an awful extension for it to have that much effect. Most things you can mess with the big effect is like, oh a thousand tokens ended up in the ~system prompt, or 10% extra or fewer work based on extra tool calls or churning through thinking or whatever. Harness stuff if it's 8x worse that's like, it's fucked and broken, something went _wrong_.

crossroadsguy 8d ago

I don't know what you tried but I tried to do something in Pi what I'd have usually done in Claude and it devoured tokens (much more than Claude) and I wasn't even close to finishing the task. Mostly because the barebones Pi was/is very inefficient at anything with any weight. Then I began to customise it to be as good as Claude but eat less taken. I got tired and I had not even scratched the surface. Gave up. I finally realised, at least for me, Pi's best use case is - strip even the little "extra" Pi comes/starts with and then use it just like that if you have a task/work that is appropriate for that bareness.