43 papers, in module order

every paper behind the code

  1. free pdfAutomatic Differentiation in Machine Learning: a Survey

    Atılım Güneş Baydin, Barak A. Pearlmutter, Alexey Andreyevich Radul, Jeffrey Mark Siskind

    automatic differentiation

  2. paywalledRidge Regression: Biased Estimation for Nonorthogonal Problems

    Arthur E. Hoerl, Robert W. Kennard

    linear regression, ridge and lasso

  3. free pdfRegression Shrinkage and Selection via the Lasso

    Robert Tibshirani

    linear regression, ridge and lasso

  4. paywalledThe Regression Analysis of Binary Sequences

    David R. Cox

    logistic regression

  5. free pdfNearest Neighbor Pattern Classification

    Thomas M. Cover, Peter E. Hart

    k-nearest neighbours

  6. free pdfA Comparison of Event Models for Naive Bayes Text Classification

    Andrew McCallum, Kamal Nigam

    naive bayes

  7. free pdfInduction of Decision Trees

    J. Ross Quinlan

    decision trees

  8. bookClassification and Regression Trees

    Leo Breiman, Jerome H. Friedman, Richard A. Olshen, Charles J. Stone

    decision trees

  9. free pdfBagging Predictors

    Leo Breiman

    bagging and random forests

  10. free pdfRandom Forests

    Leo Breiman

    bagging and random forests

  11. free pdfA Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting

    Yoav Freund, Robert E. Schapire

    boosting: adaboost and gradient boosting

  12. free pdfGreedy Function Approximation: A Gradient Boosting Machine

    Jerome H. Friedman

    boosting: adaboost and gradient boosting

  13. free pdfA Training Algorithm for Optimal Margin Classifiers

    Bernhard E. Boser, Isabelle M. Guyon, Vladimir N. Vapnik

    support vector machines

  14. free pdfSupport-Vector Networks

    Corinna Cortes, Vladimir Vapnik

    support vector machines

  15. free pdfSequential Minimal Optimization: A Fast Algorithm for Training Support Vector Machines

    John C. Platt

    support vector machines

  16. paywalledLeast Squares Quantization in PCM

    Stuart P. Lloyd

    k-means clustering

  17. free pdfk-means++: The Advantages of Careful Seeding

    David Arthur, Sergei Vassilvitskii

    k-means clustering

  18. free pdfOn Lines and Planes of Closest Fit to Systems of Points in Space

    Karl Pearson

    principal component analysis

  19. paywalledAnalysis of a Complex of Statistical Variables into Principal Components

    Harold Hotelling

    principal component analysis

  20. paywalledThe Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain

    Frank Rosenblatt

    perceptron, mlp and backpropagation

  21. free pdfLearning Representations by Back-Propagating Errors

    David E. Rumelhart, Geoffrey E. Hinton, Ronald J. Williams

    perceptron, mlp and backpropagation

  22. free pdfEfficient BackProp

    Yann LeCun, Léon Bottou, Genevieve B. Orr, Klaus-Robert Müller

    activation functions

  23. paywalledProbabilistic Interpretation of Feedforward Classification Network Outputs, with Relationships to Statistical Pattern Recognition

    John S. Bridle

    activation functions

  24. free pdfTraining Stochastic Model Recognition Algorithms as Networks can Lead to Maximum Mutual Information Estimation of Parameters

    John S. Bridle

    activation functions

  25. free pdfRectified Linear Units Improve Restricted Boltzmann Machines

    Vinod Nair, Geoffrey E. Hinton

    activation functions

  26. free pdfDeep Sparse Rectifier Neural Networks

    Xavier Glorot, Antoine Bordes, Yoshua Bengio

    activation functions

  27. free pdfRectifier Nonlinearities Improve Neural Network Acoustic Models

    Andrew L. Maas, Awni Y. Hannun, Andrew Y. Ng

    activation functions

  28. free pdfDelving Deep into Rectifiers: Surpassing Human-Level Performance on ImageNet Classification

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, Jian Sun

    activation functions · training techniques: init, dropout, batch norm, schedules, early stopping

  29. free pdfFast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)

    Djork-Arné Clevert, Thomas Unterthiner, Sepp Hochreiter

    activation functions

  30. free pdfGaussian Error Linear Units (GELUs)

    Dan Hendrycks, Kevin Gimpel

    activation functions

  31. free pdfSearching for Activation Functions

    Prajit Ramachandran, Barret Zoph, Quoc V. Le

    activation functions

  32. paywalledSome Methods of Speeding up the Convergence of Iteration Methods

    Boris T. Polyak

    optimizers: momentum, rmsprop, adam, adamw

  33. free pdfOn the Importance of Initialization and Momentum in Deep Learning

    Ilya Sutskever, James Martens, George Dahl, Geoffrey Hinton

    optimizers: momentum, rmsprop, adam, adamw

  34. lecture slidesLecture 6e: rmsprop: Divide the gradient by a running average of its recent magnitude

    Geoffrey Hinton, Nitish Srivastava, Kevin Swersky

    optimizers: momentum, rmsprop, adam, adamw

  35. free pdfAdam: A Method for Stochastic Optimization

    Diederik P. Kingma, Jimmy Ba

    optimizers: momentum, rmsprop, adam, adamw

  36. free pdfDecoupled Weight Decay Regularization

    Ilya Loshchilov, Frank Hutter

    optimizers: momentum, rmsprop, adam, adamw

  37. free pdfUnderstanding the Difficulty of Training Deep Feedforward Neural Networks

    Xavier Glorot, Yoshua Bengio

    training techniques: init, dropout, batch norm, schedules, early stopping

  38. free pdfDropout: A Simple Way to Prevent Neural Networks from Overfitting

    Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, Ruslan Salakhutdinov

    training techniques: init, dropout, batch norm, schedules, early stopping

  39. free pdfBatch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

    Sergey Ioffe, Christian Szegedy

    training techniques: init, dropout, batch norm, schedules, early stopping

  40. free pdfCyclical Learning Rates for Training Neural Networks

    Leslie N. Smith

    training techniques: init, dropout, batch norm, schedules, early stopping

  41. free pdfSGDR: Stochastic Gradient Descent with Warm Restarts

    Ilya Loshchilov, Frank Hutter

    training techniques: init, dropout, batch norm, schedules, early stopping

  42. free pdfEarly Stopping - But When?

    Lutz Prechelt

    training techniques: init, dropout, batch norm, schedules, early stopping

  43. free pdfGradient-Based Learning Applied to Document Recognition

    Yann LeCun, Léon Bottou, Yoshua Bengio, Patrick Haffner

    convolutional neural networks