6 ms·
Given the new architecture, speeds are harder to estimate. Max would be ~40 tok/s.
by rohansood15 21d ago
Given the new architecture, speeds are harder to estimate. Max would be ~40 tok/s.
- lxkarthi 21d agoI can run Deepseek flash 0731 version (ds4, esl3) on single DGX spark. getting around ~20 tok/s. It's great. Quantized version of this model would probably run on the DGX spark. I am excited to wait for quantized models that fits in single DGX spark.