6 ms·
Scale of data we see on our HPC, it is way better performance per £/$ to use Lustre mounted over fast network. Would spend far too much time shifting data other
by NGRhodes 1y ago
Scale of data we see on our HPC, it is way better performance per £/$ to use Lustre mounted over fast network. Would spend far too much time shifting data otherwise. Local storage should be used for tmp and scratch purposes.
- snickerdoodle12 1y agoThe docker image is a scratch purpose.
- trueismywork 1y agoImagine copying 8gb image to 96000 ranks over network
- 0xbadcafebee 1y agoIt's called caching layers bruv, container images do it. Plus you can stagger registries in a tiered cache per rack/cage/etc. OTOH, constantly re-copying the same executable over and over every time you execute or access it over a network filesystem wastes bandwidth and time, and a network filesystem cache is both inefficient and runs into cache invalidation issues.