6 ms·
I've been experimenting with routing requests between different models depending on the task complexity to keep costs down. It's surprising how much you can sav
by johalmed 6mo ago
I've been experimenting with routing requests between different models depending on the task complexity to keep costs down. It's surprising how much you can save just by defaulting to a smaller model for simple extraction tasks and reserving the heavy hitters only for complex reasoning. The tooling around this is definitely getting better, but tracking the actual spend per model in real-time is still a bit of a headache for side projects.