15 ms·
What's the speed on DeepSeek Flash? And what provider?
by okdood64 3mo ago
What's the speed on DeepSeek Flash? And what provider?
- ForHackernews 3mo agoFast enough? I signed up directly with https://platform.deepseek.com/ https://platform.deepseek.com/ because it was the cheapest I could find. I use both Anthropic and Deepseek models via the VS Code copilot plugin https://github.com/Vizards/deepseek-v4-for-copilot https://github.com/Vizards/deepseek-v4-for-copilot
- embedding-shape 3mo agods4: prefill: 150.18 t/s, generation: 48.47 t/s together with https://github.com/antirez/ds4 https://github.com/antirez/ds4 + RTX Pro 6000, local provider :) I guess you could rent one too, get similar speed. https://gist.github.com/embedding-shapes/eceefa1b2699dca448f9af22e995cdeb https://gist.github.com/embedding-shapes/eceefa1b2699dca448f... The weights I've tried is the "DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix" ones which fits just about within 96GB VRAM. With some tuning, I've managed to get it to get up to ~60 t/s, I'm sure there is other things to do there too :)