4 ms·

This is just an API level change right? The model itself should be the same I think.

• k__ 2mo ago
  As I understand it, they would have to train a whole knew model to hard cap it's context to different lengths. That would be cheaper to train and had cheaper inf, but still a huge investment. So I'd guess it's API level.
  • gpugreg 2mo ago
    From the Kimi K3 technical report: Kimi K3 supports a context window of up to 1 million tokens. We achieve this through extending the context window progressively as training proceeds, following a four-stage curriculum. The window grows from 8K to 64K tokens during pre-training, and from 256K to 1M tokens during the cooldown phase. https://arxiv.org/pdf/2607.24653 https://arxiv.org/pdf/2607.24653 page 12
  • dannyw 2mo ago
    You absolutely don't need to 'retrain' to reduce your context window. In vLLM it is an inference parameter. Smaller context window, smaller KV, less RAM needed to serve the same volume of requests. With a lot of architectures, you technically don't need to retrain to extend the context window either; e.g. RoPE scaling; but performance is typically crap.