Posts tagged as regularization
Dropout and Regularization
First mention of dropout is found in (Hinton et al. 2012). That paper talks about preventing feature correlation in neural networks. Dropout was applied successfully in (Krizhevsky et al. 2012) after which it gained widespread popularity. It was shown to be effective in Recurrent Neural Networks for the first time in (Zaremba et al. 2014). Historically, neural network pruning was an effective way to prevent overfitting of neural networks (Hassibi and Stork 1993; LeCun et al. 1990). These methods used ideas from perturbation theory to minimize the change in second order gradients (hessian)....Posted on: 2018-07-04, in Category: research
