26 ms·
the problem is multinode runs that communicate through the network
by jfkfif 2y ago
the problem is multinode runs that communicate through the network
- josh-sematic 2y agoYes, which is especially important for training. Getting good GPU interconnect can be really important for training large models.
- freeone3000 2y agoMultinode runs don’t communicate through the network in a DGX configuration. NVlink allows for RDMA over direct infiniband. No need for network here.
- tomoyoirl 2y agoInfiniband is a network too… But even if we set that aside you’ll get access to your data over a network connection because these are expensive nodes running batch jobs with finite disk space, not personal workstations.
- freeone3000 2y agoYea ofc. Nvidia has for mellanox infiniband, nvlink external, and even mellanox ethernet pci-to-pci, no need to involve the CPU. nvidia-docker has a few mods to support this too.