7 ms·

What hardware do you use, out of curiosity?

• jychang 1y ago
  In the current era of MoE models, the system RAM memory bandwidth determines your speed more than the GPU does.