5 ms·
Aren't you describing why they use mixture of experts? Where a sub-set of weights are activated depending on the query?
by frde_me 6mo ago
Aren't you describing why they use mixture of experts? Where a sub-set of weights are activated depending on the query?