nostrowski

5 ms·

1. 1.▲
   by nostrowski 2y ago

   Two things I'm curious to know: 1. How many tokens can 'traditional' models (e.g. Mistral's 8x7B) fit on a single 80GB GPU? 2. How does quantization affect the single transformer layer in the stack? What are the performa
2. 2.▲
   by nostrowski 3y ago

   Unfortunate, but impressed by how the Mintlify team is handling it.
3. 3.▲
   by nostrowski 3y ago

   This will be in a future history book under a chapter titled "the beginning of the end"