5 ms·

I configured a dual DGX Spark cluster, and it's certainly "good enough" for my agentic and coding needs.

by mv4 4mo ago

datadrivenangel 4mo ago
what models are you using on that? My experiences with apple hardware have convinced me that it is not really good enough for coding locally.
- mv4 4mo ago
  Minimax M2.7.
- irishcoffee 4mo ago
  It isn’t the models, it’s the closed api and the tooling associated with it. It’s driving me crazy how not-talked-about this is.
  - datadrivenangel 4mo ago
    As in the coding harnesses?
    - irishcoffee 4mo ago
      If I could leverage the same closed api VSCode uses, the entire moat is drained. What you call harnesses I call… bullshit?
  - klausa 4mo ago
    You can point both Codex and Claude Code at a local model and they'll work just fine. Codex even explicitly supports that as a feature! [1] With a nice UI on top, for the desktop app too: [2] [1]: https://developers.openai.com/codex/config-advanced#custom-model-providers https://developers.openai.com/codex/config-advanced#custom-m... [2]: https://docs.ollama.com/integrations/codex-app https://docs.ollama.com/integrations/codex-app
- girvo 4mo ago
  DeepSeek v4 Flash, various quantised versions of Kimi K2.6, MiniMax 2.7, Qwen 3.5 “full sized, with a dual spark setup you can fit some decent setups on here My single spark has me running Qwen 3.6 27B and antirez’s specially quantised DeepSeek v4 Flash (which is shockingly impressive)
  - Tepix 4mo ago
    Kimi K2.6 does not run well on 256GB.
    - girvo 4mo ago
      True, I might be thinking of some of the communities four-Spark clusters for it; it’s already int4 right?
      - Tepix 4mo ago
        Yeah, the default quants are 595GB. Even four Sparks would require a quant lower than 4bit
    - zozbot234 4mo ago
      Have you tried it? It would be slow for sure, but the main limitation AIUI would actually be storing the context in RAM - models like Kimi and GLM have high demands there which limit your ability to get meaningful aggregate throughput via large batches.
      - Tepix 4mo ago
        No need to try really. 1100b weights with 256GB RAM that‘s less than 1.8 bits per weight if you want a little bit of context. How is that supposed to give good results?

I configured a dual DGX Spark cluster, and it's certainly "good enough" for my agentic and coding needs.

by mv4 4mo ago

I configured a dual DGX Spark cluster, and it's certainly "good enough" for my agentic and coding needs.

what models are you using on that? My experiences with apple hardware have convinced me that it is not really good enough for coding locally.

mv4 4mo ago
Minimax M2.7.
irishcoffee 4mo ago
It isn’t the models, it’s the closed api and the tooling associated with it. It’s driving me crazy how not-talked-about this is.
- datadrivenangel 4mo ago
  As in the coding harnesses?
  - irishcoffee 4mo ago
    If I could leverage the same closed api VSCode uses, the entire moat is drained. What you call harnesses I call… bullshit?
- klausa 4mo ago
  You can point both Codex and Claude Code at a local model and they'll work just fine. Codex even explicitly supports that as a feature! [1] With a nice UI on top, for the desktop app too: [2] [1]: https://developers.openai.com/codex/config-advanced#custom-model-providers https://developers.openai.com/codex/config-advanced#custom-m... [2]: https://docs.ollama.com/integrations/codex-app https://docs.ollama.com/integrations/codex-app
girvo 4mo ago
DeepSeek v4 Flash, various quantised versions of Kimi K2.6, MiniMax 2.7, Qwen 3.5 “full sized, with a dual spark setup you can fit some decent setups on here My single spark has me running Qwen 3.6 27B and antirez’s specially quantised DeepSeek v4 Flash (which is shockingly impressive)
- Tepix 4mo ago
  Kimi K2.6 does not run well on 256GB.
  - girvo 4mo ago
    True, I might be thinking of some of the communities four-Spark clusters for it; it’s already int4 right?
    - Tepix 4mo ago
      Yeah, the default quants are 595GB. Even four Sparks would require a quant lower than 4bit
  - zozbot234 4mo ago
    Have you tried it? It would be slow for sure, but the main limitation AIUI would actually be storing the context in RAM - models like Kimi and GLM have high demands there which limit your ability to get meaningful aggregate throughput via large batches.
    - Tepix 4mo ago
      No need to try really. 1100b weights with 256GB RAM that‘s less than 1.8 bits per weight if you want a little bit of context. How is that supposed to give good results?