7 ms·
Hinton's DropOut [1] and Wan's DropConnect [2] have ameliorated some of the overfitting issues present in traditional NN's. In fact, DropConnect in conjunction
by billderose 12y ago
Hinton's DropOut [1] and Wan's DropConnect [2] have ameliorated some of the overfitting issues present in traditional NN's. In fact, DropConnect in conjunction with deep learning are responsible for new records being set on classical datasets such as MNIST.
[1] http://arxiv.org/pdf/1207.0580.pdf http://arxiv.org/pdf/1207.0580.pdf
[2] http://cs.nyu.edu/~wanli/dropc/ http://cs.nyu.edu/~wanli/dropc/
- im3w1l 12y agoIt's pretty funny, I saw DropConnect described in a stackoverflow answer that predated the paper you reference. It was an incorrect answer on how to do dropout. I shall try to find it tomorrow.
- agibsonccc 12y agoDropout is actually a knob on any neural network. These are used in image recognition as well as text and other areas. The fuzzing creates a very similar effect to convolutional nets where it can learn different poses of an image.