46 ms·
It also comes down to inference speed, not "can I run this". 8-bit quant is quite a bit slower on an M5 Pro.
by bitexploder 3mo ago
It also comes down to inference speed, not "can I run this". 8-bit quant is quite a bit slower on an M5 Pro.
46 ms·