6 ms·
50% cache hit is really low - in a standard agentic loop you should expect like 99%+ cache hit percentage (which should also lower that $12.50 to like a couple
by p1necone 1mo ago
50% cache hit is really low - in a standard agentic loop you should expect like 99%+ cache hit percentage (which should also lower that $12.50 to like a couple of $ for the same amount of tokens).
If you're using a customised harness you should make sure you don't have something that's e.g. changing your system prompt on some requests or rewriting history - it can be tempting to do stuff like strip old thinking tokens or compact tool call results to reduce context size but it's a trap - you want to never change history because of how cheap cache is, even more so with deepseek because their cache hit pricing is so low compared to most other models.
- Lalabadie 1mo agoIn my experience, that's the OpenRouter tax. Even a session that does everything right to remain sticky ends up getting moved between providers on a few requests, which bills you the full context as input every time the switch happens. I assume it's done as load balancing/latency mitigation, but it's put me off of OpenRouter for my use cases (limited use, limited need for changing models).
- grono 1mo agoWhy not pin to specific openrouter provider and disable fallback?
- julianz 1mo agoThere is only one provider for this model, so shouldn't be running into that.
- p1necone 1mo agoThis has not been my experience. Generally I do pin to 1 provider, or 1 provider with a couple fallbacks (especially with deepseek - most providers are 10x the cached token price compared to deepseek themselves), but even when I don't I still usually see 99%+ cache hit percentage. Specifically using pi with various ad-hoc customisations (that I was careful not to break prompt caching with).
- irthomasthomas 1mo agothen what is the point of using operouter for this model? Just use the deepseek API and save the 5% fee on top of the better caching rate.
- FridgeSeal 1mo agoBecause they don’t want to sign up for 10 different providers and subscriptions/etc, especially if some models are just going to receive light, or rare usage?
- wut42 1mo agoIt is a pain from openRouter if you don't define your providers correctly, but for DeepSeek, surely not- the weights aren't released yet and there's only one provider, DeepSeek.
- Lalabadie 1mo agoWith Deepseek as the provider, there's no issue of course, but that means you don't filter providers for data retention, and you could also choose direct API use with them at that point.
- wut42 1mo agoFallback are still very useful and won't poison much your cache hits too much if the provider is down anyway.
- stingraycharles 1mo agoThat’s a very heavy tax tbh.
- seunosewa 1mo agoYou can set it up to always use the official provider.
- isqueiros 1mo agoSeems like pro 0813 is exclusively served by Deepseek themselves at the moment so I wouldn't say that's the case?
- monster_truck 1mo agoThank you for the advice but it looks like it's an issue with Deepseek's dashboard, I think they are calculating the value incorrectly. This prompted me to dig back through the logs, it seems to be closer to 96% at its absolute worst and 99.3 at best. I only sell to openrouter, would much rather support Deepseek directly.