5 ms·
mistral 7b v0.2 supports 32k
by kiraaa 3y ago
mistral 7b v0.2 supports 32k
- brucethemoose2 3y agoThis is a good point actually, and an underappreciated fact. I think so many people (including me) effectively ignored Mistral 0.1's sliding window that few realized 0.2 instruct is native 32K.
- tarruda 3y agoMixtral 8x7B has 32k context. Mistral 7b instruct 0.2 is just an instruct fine tune of Mistral 7b and stays with a 8k context.