14 ms·
Quantization and speculative decoding unlocked major savings for our smaller models. Still chasing that ideal cost/performance ratio.
by copperwire 15d ago
Quantization and speculative decoding unlocked major savings for our smaller models. Still chasing that ideal cost/performance ratio.