8 ms·
Things like the number of GPU layers, whether to use flash attention or not, enabling MTP, playing with context size.
by drittich 1mo ago
Things like the number of GPU layers, whether to use flash attention or not, enabling MTP, playing with context size.