6 ms·
The math isn't super difficult. Some books will try to throw a mess of differential equations at you, but some simple calculus is all you need for backpropagati
by vippy 3y ago
The math isn't super difficult. Some books will try to throw a mess of differential equations at you, but some simple calculus is all you need for backpropagation.
- quickthrower2 3y agoI have been through the math thanks to the youtube videos by A. Karpathy. Deriving some of the differentials, e.g. for batchnorm seems fairly hard (hard as in slogging through something with many steps where you can't make a mistake at any step). But the principles are quite simple - I think by design. If they were hard to compute or reason about then the neural net wouldn't work very well!
- sva_ 3y agoDoing the compute efficiently, especially from Python, is the tricky part.