5 ms·
That is not entirely accurate. The Science paper described how to (pre)train a deep belief net by training a sequence of RBMs. Contrastive divergence for RBM tr
by gdahl 14y ago
That is not entirely accurate. The Science paper described how to (pre)train a deep belief net by training a sequence of RBMs. Contrastive divergence for RBM training (and more generally products of experts) was described in 2002 in "Training Products of Experts by Minimizing Contrastive Divergence" http://www.cs.toronto.edu/~hinton/absps/nccd.pdf http://www.cs.toronto.edu/~hinton/absps/nccd.pdf
- karpathy 14y agodoh, not very carefully worded now that I'm re-reading my answer, you're right of course. Well, at least we're slowly converging on the right answer over several comments :)