23 ms·
This is where MoEs shine though. You don't need all experts in memory at once. Diffusion inference doesn't have sparse inference.
by m00x 2mo ago
This is where MoEs shine though. You don't need all experts in memory at once. Diffusion inference doesn't have sparse inference.