5 ms·
Indeed they are all lossy. Not sure how much they contribute to the quality loss in long context though. I got a 700k session with DSV4 Pro (official API), and
by throwdbaaway 2mo ago
Indeed they are all lossy. Not sure how much they contribute to the quality loss in long context though. I got a 700k session with DSV4 Pro (official API), and the model was still coherent and didn't make any tool call error.
- stingraycharles 2mo agoThat’s a low bar though, and the least I would expect.
- throwdbaaway 2mo agoWell I wouldn't call it a low bar, since some of the edits were quite complex. And 1M context in less than 6GB of VRAM is truly impressive, but somehow this gets way less attention than the crappy turbo quant from Google.
- scope2093 2mo agoI'd like to understand this please. Why would the 1M context be kept in VRAM if you're using DSV4 Pro through the API? Or did you refer to different sessions?
- throwdbaaway 2mo agoDifferent sessions. With https://github.com/fairydreaming/llama.cpp/tree/dsv4 https://github.com/fairydreaming/llama.cpp/tree/dsv4, 1M context with DSV4 Flash takes less than 6GB of VRAM. I can't run DSV4 Pro, but it should take less than 9GB of VRAM for 1M context, based on the numbers shared in https://arxiv.org/html/2606.19348v1 https://arxiv.org/html/2606.19348v1.
- scope2093 2mo agoThank you for the links/docs. I'm quite excited to try it myself.