4 ms·
You transfer context from the kv cache of model A to the kv cache of model B
by FrancescoMassa 2mo ago
You transfer context from the kv cache of model A to the kv cache of model B
- verdverm 2mo agoyou cannot do this with direct byte copy, you have to reprocess all input to rebuild the cache in model B, it's exactly like waiting hours to send a new message to an existing session and your cache has expired tl;dr - you will not get the 90% discount the first time you change models and send a message this is why I am saying you cannot "transfer" kv cache, it's just like changing models in your favorite harness and that requires reprocessing the input from scratch to build a kv cache for the model
- FrancescoMassa 2mo agoit works exactly like that...that's why you're seeing the notice con claude code when switching models