Regarding 2: I saw a talk yesterday by Alex Hernández-García in which he showed that dropout and weight decay can be replaced with data augmentation in many cas

Regarding 2: I saw a talk yesterday by Alex Hernández-García in which he showed that dropout and weight decay can be replaced with data augmentation in many cases. The relevant paper by him and König is https://arxiv.org/abs/1806.03852v2 https://arxiv.org/abs/1806.03852v2 edit: changed to /abs/ instead of /pdf/ link