5 ms·It’s cheaper sure, but it’s very slow. It’s not a drop in replacementby _aavaa_ 19d agoIt’s cheaper sure, but it’s very slow. It’s not a drop in replacementnatrys 19d agoI think we don't have a good draft model for better speculative decoding yet (e.g. DFlash 2). Once we do, it will be faster._aavaa_ 19d agoIt very well could be faster, but right now it isn’t.