The Regression Analysis of Binary Sequences
David R. Cox
Journal of the Royal Statistical Society, Series B 20(2):215-242 (with discussion)
about this write-up
No legal free copy was found. This write-up is from secondary sources and standard accounts of the history of logistic regression, not from a reading of the original PDF.
the original →the background
Yes/no outcomes were already studied in drug testing: give a dose, count how many respond. The standard tool was the probit model, which uses the bell curve's S-shape. Others had argued for the logistic S-curve and coined the word "logit".
Cox turned this into a general regression method for yes/no data: any number of inputs, and outcomes that may come in a sequence over time.
the model
For each trial, with success probability θ and input x:
the main argument: the maths gets simple
The logistic and probit curves look almost the same, so data alone can't easily choose
between them. Cox's point: with the logistic curve, the data only matters through
two simple sums, Σy and Σxy. The log-likelihood is
That makes fitting straightforward, and it allows exact tests. The probit model has no such shortcut.
"sequences"
The title refers to yes/no outcomes observed in order, where the chance can depend on time or on earlier outcomes (for example, on whether the previous trial succeeded).
why it lasted
- Each weight is a log odds ratio, which is easy to explain.
- The loss has one bottom and no traps, so Newton's method finds it reliably.
- It became the model for yes/no data in the wider family of generalised linear models.
- In machine learning it's the sigmoid output with cross-entropy loss, and softmax for more than two classes.
in this repo
LogisticRegression(solver="newton")is the classical fit, and it reports standard errors (se_W,se_b).- The logistic demo shows those standard errors match the real spread of the weights over 300 simulated datasets.