5 ms·
Maybe I am blinded by my own use case, but I find the caching pricing and strategy (since different providers use a different implementation of caching as well
by Fripplebubby 1y ago
Maybe I am blinded by my own use case, but I find the caching pricing and strategy (since different providers use a different implementation of caching as well as different pricing) to be a major factor rather than just the "raw" per token cost, and that is missing here, as well as on the Simon Willison site [1]. Do most people just not care / not use caching that much that it matters?
[1] https://llm-prices.com/ https://llm-prices.com/
- MattSayar 1y agoI know at least a couple LLM providers will do some caching for you automatically now, which muddies the waters a bit. [0] [0] https://developers.googleblog.com/en/gemini-2-5-models-now-support-implicit-caching/ https://developers.googleblog.com/en/gemini-2-5-models-now-s...