7 ms·
The idea is that smarter models might use fewer turns to accomplish the same task - reducing the overall token usage Though, from my limited testing, the new m
by dktp 5mo ago
The idea is that smarter models might use fewer turns to accomplish the same task - reducing the overall token usage
Though, from my limited testing, the new model is far more token hungry overall
- manmal 5mo agoWell you‘ll need the same prompt for input tokens?
- httgbgg 5mo agoOnly the first one. Ideally now there is no second prompt.
- manmal 5mo agoAre you aware that every tool call produces output which also counts as input to the LLM?
- squeaky-clean 5mo agoAre you aware that a lot of model tool calls are useless and a smarter model could avoid those? Are you aware that output tokens are priced 5x higher than input tokens?
- manmal 5mo ago> a lot of model tool calls are useless That’s just wrong. File reads, searches, compiler output, are the top input token consumers in my workflow. None of them can be removed. And they are the majority of my input tokens. That’s also why labs are trying to make 1M input work, and why compaction is so important to get right. Regarding output - yes, but that wasn’t the topic in this thread. It’s just easier to argue with input tokens that price has gone up. I have a hunch the price for output will go up similarly, but can’t prove it. The jury’s out IMO: https://news.ycombinator.com/item?id=47816960 https://news.ycombinator.com/item?id=47816960
- httgbgg 5mo agoThis has no bearing on my comment. The point is that a better model avoids dozens of prompts and tool calls by making fewer CORRECT tool calls, with the user needing no more prompts. I’m surprised this is even a question; obviously a better prompter has the same properties and it’s not in dispute?