4 ms·for posterity, since shown that is it actually MoE > 21B parameters with 3.6B active parametersby nivvis 1y agofor posterity, since shown that is it actually MoE > 21B parameters with 3.6B active parameters