lustre-fan

6 ms·

1. 1.▲
   by lustre-fan 7mo ago

   FreeBSD doesn't have support for CUDA, many third party network fabrics and accelerators, Lustre, and other high performance storage solutions. It makes FreeBSD for frontier machines a non-starter. FreeBSD adoption for AI trainer/
2. 2.▲
   by lustre-fan 1y ago

   > do you have something set to monitor HN for mentions of Lustre? Nothing, beside browsing hackernews a bit too much. > "But Lustre doesn't support NVMeoF directly. It uses a custom protocol." To be specific, Lustre is
3. 3.▲
   by lustre-fan 1y ago

   Yeah, Lustre supports EFA as a network transit between a Lustre client and a Lustre server. It's lnet/klnds/kefalnd/ in the Lustre tree. But Lustre doesn't support NVMeoF directly. It uses a custom protocol. And nei
4. 4.▲
   Getting the Lustre Filesystem Upstream(lwn.net)

   4 points by lustre-fan 1y ago|0 comments
5. 5.▲
   by lustre-fan 1y ago

   I work on Lustre development. Lustre uses a lot of kernel symbols not covered by the kABI stability guarantee and we already need to maintain configure checks for all of the other kernels (SuSe, Ubuntu, mainline, etc) that don't offer
6. 6.▲
   Amazon FSx for Lustre Now Supports Elastic Fabric Adapter and GPUDirect Storage(aws.amazon.com)

   1 points by lustre-fan 2y ago|0 comments
7. 7.▲
   by lustre-fan 2y ago

   I think FSx/Lustre would be the simplest assuming you're using Linux instances. You can definitely scale to 256GB/s or more. Plus, you can automatically hydrate the dataset from s3. Lustre is common for HPC and ML/AI, so