natechlin

7 ms·

1. 1.▲
   by natechlin 6mo ago

   Prefill - module computes KV cache over input toks, up to the last token in your input (the 'prompt'), at which point it can then begin - Decode - the model chooses a new token to append to the end of the current token list (i.e.
2. 2.▲
   Managing a million dollar pool with 15k lines of Go(nat-echlin.com)

   5 points by natechlin 8mo ago|1 comments