6 ms·
yeah but using the rule of thumb (I see floating in the Internet) which is sqrt(#params * #active params) which would give sqrt(284B*13B) = 60B, so deepseek sho
by K0IN 1mo ago
yeah but using the rule of thumb (I see floating in the Internet) which is sqrt(#params * #active params) which would give sqrt(284B*13B) = 60B, so deepseek should perform better.
- apitman 1mo agoSeems like that equation really should weight total params and active params differently