7 ms·
Silent Data Corruptions: The Boogeyman of LLM Training
- ibeitia 3y agoFascinating read!
- osavant 3y agoSuper interesting, thanks for putting this together
- ejro 3y agoInteresting. This is probably a universal problem for large model training but not being discussed enough.
- adeptlo 3y agoSuper interesting problem that's affecting more people than they probably realize.
- auraham 3y agoInteresting post. It would be much better if the author included a few code snippets to show how to identify the failing GPU during training.