8 ms·
Lookahead Sparse Attention should be playing a big role as well, as it dramatically slashes memory consumption.
by _0ffh 3mo ago
Lookahead Sparse Attention should be playing a big role as well, as it dramatically slashes memory consumption.
loading thread…