8 ms·
Different sessions. With https://github.com/fairydreaming/llama.cpp/tree/dsv4 https://github.com/fairydreaming/llama.cpp/tree/dsv4, 1M context with DSV4 Flash t
by throwdbaaway 2mo ago
Different sessions. With https://github.com/fairydreaming/llama.cpp/tree/dsv4 https://github.com/fairydreaming/llama.cpp/tree/dsv4, 1M context with DSV4 Flash takes less than 6GB of VRAM. I can't run DSV4 Pro, but it should take less than 9GB of VRAM for 1M context, based on the numbers shared in https://arxiv.org/html/2606.19348v1 https://arxiv.org/html/2606.19348v1.
- scope2093 2mo agoThank you for the links/docs. I'm quite excited to try it myself.