6 ms·
You're using it on low, that's why. There's a huge difference in performance from low to max effort.
by theogravity 1mo ago
You're using it on low, that's why. There's a huge difference in performance from low to max effort.
- zacksiri 1mo agoI’m comparing same / similar settings between models. I can’t use high on one and low on others it’s not a fair test. Not sure why I was downvoted. But seems the downvoter is quick to downvote anything that doesn’t fit the narrative they’re looking for. I’m just reporting my findings.
- theogravity 1mo agoI did not downvote you. I think it was the way you wrote the comment which made it seem like it couldn't perform in general.
- kgeist 1mo agoI'm not sure it's a fair test either to compare the "low" setting of one model with the "low" setting of another. They're completely different settings that just happen to have the same name.
- lejalv 1mo agoComparing at similar thinking hasn't much value. You can compare the tiers that have the closest price, that would be more interesting.
- zacksiri 1mo agoYes, I think a proper comprehensive test would be a better judge of the outcome. I may do round 2 given my first batch of models is already outdated.
- Palmik 1mo agoLow, High and Max, obviously, can't be compared across models. They only mean the model is likely to spend less reasoning effort (~output tokens) with Low than High on the same, *single shot* task. But even in this very post, you can see that Max was actually cheaper than High. If you are using API, you should be comparing based on end-to-end cost or speed or whatever blend of those two matches your cost/time budget.