← module 11 · perceptron, mlp and backpropagation
paywalled

The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain

Frank Rosenblatt

Psychological Review 65(6):386-408

about this write-up

Officially paywalled (a copy exists on a university course page, of unclear legal status). This write-up is from secondary sources and standard histories, not from a reading of the original.

the original →

what the paper was

A psychology paper about the brain, not a computer-science paper. Rosenblatt asked how a physical system could store information and learn to recognise patterns, and proposed the perceptron as a model of that.

the model

His perceptron has three kinds of units:

  • sensory units: a "retina" that receives the pattern;
  • association units: each connected to a random handful of sensory units, firing when enough of them are active;
  • response units: add up the association units with weights and give the answer.

Learning changes the weights into the response units: strengthen the ones that led to a right answer, weaken the ones behind a wrong one.

The single neuron used today (one weighted sum, one threshold, and the mistake-driven update w ← w + η(y − ŷ)x) is the simplified core of this.

what followed

  • He built hardware versions, which got a lot of attention and some over-excited claims.
  • It always finishes on separable data: if a straight line can split the classes, the learning rule finds one after a limited number of mistakes. The tidy bound on that number came later, from Novikoff.
  • Minsky and Papert later analysed single-layer perceptrons carefully and showed what they can't do, such as XOR. This is often linked to a slowdown in neural-network research, though the real story is more complicated.
  • Multi-layer networks trained with backpropagation removed the one-layer limit. Rosenblatt had discussed multi-layer perceptrons himself, but had no good way to train the hidden layers.

in this repo

  • Perceptron is the single-neuron learning rule, with mistakes logged per pass.
  • The MLP demo shows it finishing on separable data and never settling on XOR, which a network with one hidden layer then solves.