66 ms·
Essentially, yes. The DGX Sparks, at 128 gigs for ~$4700 are one of the cheapest ways to get enough high-ish speed memory cobbled together to run one of the mor
by ericd 13d ago
Essentially, yes. The DGX Sparks, at 128 gigs for ~$4700 are one of the cheapest ways to get enough high-ish speed memory cobbled together to run one of the more capable open weight models to run at home/for a small biz. This switch lets you combine the memory of 2-4 of them.
- bitbckt 13d ago4-8 of them. 2 Sparks can operate without a switch.
- ericd 13d agoWell, this one only has 4 ports, would you daisy chain them somehow to get to 8? Or bridge three of these switches?
- dgl 13d agoThey are QSFP56-DD which can be split using breakout cables. So it's 4 x 400GbE ports, which could in theory be split into as many as 32 x 50GbE ports. (Although whether that then ends up being an economical thing to do is a different matter.)
- ericd 13d agoAhh yeah, I've seen people using those. I was thinking you'd ideally want to connect them full speed, but it looks like they're 200gbps ports anyway, and I guess even if they were 400gpbs, you don't really need that much for cross-GPU traffic for MoE models doing tensor parallel.