8 ms·
Main issue is that token are not equivalent across provider / models. With huge disparity inside provider beyond the tokenizer model: - An image will take 10x
by pierre 1y ago
Main issue is that token are not equivalent across provider / models. With huge disparity inside provider beyond the tokenizer model:
- An image will take 10x token on gpt-4o-mini vs gpt-4.
- On gemini 2.5 pro output token are token except if you are using structure output, then all character are count as a token each for billing.
- ...
Having the price per token is nice, but what is really needed is to know how much a given query / answer will cost you, as not all token are equals.
- BonoboIO 1y agoOn gemini 2.5 pro output token are token except if you are using structure output, then all character are count as a token each for billing. Can you elaborate this? I don’t quite understand the difference.
- rsanek 1y agoI hadn't heard of this before either and can't find anything to support it on the pricing page. https://ai.google.dev/gemini-api/docs/tokens https://ai.google.dev/gemini-api/docs/tokens
- alexellman 1y agoyeah I am going to add an experiment that runs everyday and the cost of that will be a column on the table. It will be something like summarize this article in 200 words and every model gets the same prompt + article
- bigiain 1y agoFor me, and I suspect a lot of other HN readers, a comparison/benchmark on a coding task would be more useful. Something small enough that you can affordably run it every day across a reasonable range of coding focused models, but non trivial enough to be representative of day to day AI assisted coding. One other idea - for people spending $20 or $200/month for AI coding tools, a monitoring service that tracks and alerts on detected pricing changes could be something worth paying for. I'd definitely subscribe at $5/month for something like that, and I'd consider paying more, possibly even talking work into paying $20 or $30 per month.