5 ms·
DeepSeek-V4 should use only 5GB for context due to CSA and HCA, see figure here: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro https://huggingface.co/deeps
by johndough 2mo ago
DeepSeek-V4 should use only 5GB for context due to CSA and HCA, see figure here: https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro https://huggingface.co/deepseek-ai/DeepSeek-V4-Pro
But not every framework implements it properly yet.
- walrus01 2mo agoYeah, or close enough to 5GB for estimation purposes, for example a just spawned qwen 3.5 122B llama-server instance reports as: 0.07.015.888 I common_memory_breakdown_print: | - Host | 170038 = 162913 + 6740 + 384 | The 6740 is the cache size.