8 ms·
1/3rd "activated parameters", while also requiring 2x the VRAM.
by radq 2y ago
1/3rd "activated parameters", while also requiring 2x the VRAM.
- deleted 2y ago[deleted]
- YetAnotherNick 2y agoThat's the point of MoE. Sacrificing VRAM for compute/RAM bandwidth which makes it harder sell for consumer devices but easier for server devices where things are more likely to be compute or RAM bandwidth bound.