43 papers, in module order
every paper behind the code
- free pdfAutomatic Differentiation in Machine Learning: a Survey
Atılım Güneş Baydin, Barak A. Pearlmutter, Alexey Andreyevich Radul, Jeffrey Mark Siskind
- paywalledRidge Regression: Biased Estimation for Nonorthogonal Problems
Arthur E. Hoerl, Robert W. Kennard
- free pdfRegression Shrinkage and Selection via the Lasso
Robert Tibshirani
- paywalledThe Regression Analysis of Binary Sequences
David R. Cox
- free pdfNearest Neighbor Pattern Classification
Thomas M. Cover, Peter E. Hart
- free pdfA Comparison of Event Models for Naive Bayes Text Classification
Andrew McCallum, Kamal Nigam
- free pdfInduction of Decision Trees
J. Ross Quinlan
- bookClassification and Regression Trees
Leo Breiman, Jerome H. Friedman, Richard A. Olshen, Charles J. Stone
- free pdfBagging Predictors
Leo Breiman
- free pdfRandom Forests
Leo Breiman
- free pdfA Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting
Yoav Freund, Robert E. Schapire
- free pdfGreedy Function Approximation: A Gradient Boosting Machine
Jerome H. Friedman
- free pdfA Training Algorithm for Optimal Margin Classifiers
Bernhard E. Boser, Isabelle M. Guyon, Vladimir N. Vapnik
- free pdfSupport-Vector Networks
Corinna Cortes, Vladimir Vapnik
- free pdfSequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines
John C. Platt
- paywalledLeast Squares Quantization in PCM
Stuart P. Lloyd
- free pdfk-means++: The Advantages of Careful Seeding
David Arthur, Sergei Vassilvitskii
- free pdfOn Lines and Planes of Closest Fit to Systems of Points in Space
Karl Pearson
- paywalledAnalysis of a Complex of Statistical Variables into Principal Components
Harold Hotelling
- paywalledThe Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain
Frank Rosenblatt
- free pdfLearning Representations by Back-Propagating Errors
David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams
- free pdfEfficient BackProp
Yann LeCun, Léon Bottou, Genevieve B. Orr, Klaus-Robert Müller
- paywalledProbabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern Recognition
John S. Bridle
- free pdfTraining Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters
John S. Bridle
- free pdfRectified Linear Units Improve Restricted Boltzmann Machines
Vinod Nair, Geoffrey E. Hinton
- free pdfDeep Sparse Rectifier Neural Networks
Xavier Glorot, Antoine Bordes, Yoshua Bengio
- free pdfRectifier Nonlinearities Improve Neural Network Acoustic Models
Andrew L. Maas, Awni Y. Hannun, Andrew Y. Ng
- free pdfDelving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification
Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun
activation functions · training techniques: init, dropout, batch norm, schedules, early stopping
- free pdfFast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)
Djork-Arné Clevert, Thomas Unterthiner, Sepp Hochreiter
- free pdfGaussian Error Linear Units (GELUs)
Dan Hendrycks, Kevin Gimpel
- free pdfSearching for Activation Functions
Prajit Ramachandran, Barret Zoph, Quoc V. Le
- paywalledSome Methods of Speeding up the Convergence of Iteration Methods
Boris T. Polyak
- free pdfOn the Importance of Initialization and Momentum in Deep Learning
Ilya Sutskever, James Martens, George Dahl, Geoffrey Hinton
- lecture slidesLecture 6e: rmsprop: Divide the gradient by a running average of its recent magnitude
Geoffrey Hinton, Nitish Srivastava, Kevin Swersky
- free pdfAdam: A Method for Stochastic Optimization
Diederik P. Kingma, Jimmy Ba
- free pdfDecoupled Weight Decay Regularization
Ilya Loshchilov, Frank Hutter
- free pdfUnderstanding the Difficulty of Training Deep Feedforward Neural Networks
Xavier Glorot, Yoshua Bengio
training techniques: init, dropout, batch norm, schedules, early stopping
- free pdfDropout: A Simple Way to Prevent Neural Networks from Overfitting
Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov
training techniques: init, dropout, batch norm, schedules, early stopping
- free pdfBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift
Sergey Ioffe, Christian Szegedy
training techniques: init, dropout, batch norm, schedules, early stopping
- free pdfCyclical Learning Rates for Training Neural Networks
Leslie N. Smith
training techniques: init, dropout, batch norm, schedules, early stopping
- free pdfSGDR: Stochastic Gradient Descent with Warm Restarts
Ilya Loshchilov, Frank Hutter
training techniques: init, dropout, batch norm, schedules, early stopping
- free pdfEarly Stopping - But When?
Lutz Prechelt
training techniques: init, dropout, batch norm, schedules, early stopping
- free pdfGradient-Based Learning Applied to Document Recognition
Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner