6 ms·
I try to batch requests and reuse responses wherever possible.LLM API calls are awesome,but if you don't watch your usage,cost sneaks up fast.
by late_night_fix 4mo ago
I try to batch requests and reuse responses wherever possible.LLM API calls are awesome,but if you don't watch your usage,cost sneaks up fast.
- Mnexium 4mo agoAgreed, are you doing anything to take advantage of cacheing? Anything else you can share about project?