6 ms·
Prefix caching automatically detects priorly read tokens -> more fained grained checkpoints then. With prefix caching you don't have to ask for a checkpoint, it
by DelightOne 2mo ago
Prefix caching automatically detects priorly read tokens -> more fained grained checkpoints then. With prefix caching you don't have to ask for a checkpoint, its automatic.
One way or the other, a prefix is saved. Only the additional info how to continue is new.