7 ms·
Catastrophic loss of precision is, as the name implies, catastrophic in terms of the calculation context. For scientific/engineering codes, and things requirin
by hpcjoe 3y ago
Catastrophic loss of precision is, as the name implies, catastrophic in terms of the calculation context. For scientific/engineering codes, and things requiring a preservation of resolution for proper functioning, FP32 is rarely sufficient. FP64 is usually better. For ML/AI apps, resolution isn't nearly as important.
- tails4e 3y agoAbsolutely. If anything ML is migrating to lower precision types, like fp16, mx9, mx6, int8, etc.