5 ms·
The exact training run is basically impossible anyway. Randomness plays a role. Even if you fix your RNG seed, in a distributed training scenario like this one
by marcyb5st 2mo ago
The exact training run is basically impossible anyway. Randomness plays a role. Even if you fix your RNG seed, in a distributed training scenario like this one some weight updates might come at different times and be included in different update steps. Should have minimal impact on the final outcome, but would still be a different model as some of the weights will differ in the end.
- eru 2mo ago> Should have minimal impact on the final outcome, [...] I share this expectation. But this is an interesting empiric question that deserves study; even if just to confirm what 'everyone knows'.
- pcmasterr 2mo agoReproducibility isn’t something that has been considered desirable in many pipelines until recent years. Heck even ffmpeg introduce randomness when stitching together downloaded chunks from youtube. By design.
- eru 2mo agoReproducing a stationary probability distribution that subsequent runs draw from is also a kind of reproduction. And presumably for ffmpeg you can fix the random seed? Tell me more about that design, please.