5 ms·
I'm pretty sure the model is cached with the system prompt already processed. So you should only pay extra tokens.
by thomashop 1y ago
I'm pretty sure the model is cached with the system prompt already processed. So you should only pay extra tokens.