6 ms·
Probably not for a few years, you need a (maybe few) A100(s) to be able to backprop a model that big with float32.
by angry-tempest 4y ago
Probably not for a few years, you need a (maybe few) A100(s) to be able to backprop a model that big with float32.
- verdverm 4y agoiirc, they tweeted about using around 3800 in parallel