The history of machine learning How machines learned.

Shannon turned information into a mathematical quantity and established limits for representing and transmitting it through noisy channels.

Entropy and mutual information later became central tools for trees, probabilistic learning, representation analysis, and compression.

Hebb proposed that repeated co-activation could strengthen synaptic efficiency and organize neurons into cell assemblies.

The idea supplied a biological mechanism for learned representation and inspired a family of local learning rules.

Beyond the imitation game, Turing asked whether a simpler child machine could be educated through reward, punishment, mutation, and selection.

Learning appeared as an alternative to programming every adult capability by hand.

Robbins and Monro introduced a recursive procedure for finding a target when the underlying function can only be observed with noise.

Its diminishing-step update pattern became a foundation for stochastic approximation and an ancestor of iterative optimization in ML.

The Dartmouth proposal used ‘artificial intelligence’ in its title and set out research on language, abstraction, problem solving, and self-improvement.

It organized a field in which machine learning would become one major program among several.

Bellman's principle of optimality decomposed multistage decisions into recursively related value problems.

Bellman equations later underpinned value iteration, temporal-difference learning, and Q-learning.

Rosenblatt formalized a trainable brain-inspired model whose connection weights adapt from examples.

The perceptron supplied a trainable linear classifier and drew sustained attention to machines that learned from examples.

Samuel combined rote learning, search, and a scoring function improved through experience and self-play in checkers.

Samuel's checkers program gave researchers a concrete demonstration of learning through experience and self-play.

ADALINE used a linear adaptive element trained by the Widrow–Hoff or LMS rule to minimize mean-squared error.

Continuous prediction error, rather than only classification mistakes, made adaptive filtering and control practical.

Cover and Hart proved a striking guarantee for nearest-neighbor classification: storing examples and comparing a new point to them can generalize.

The work put an apparently simple instance-based method on firm statistical ground.

Vapnik and Chervonenkis related generalization to the richness of a class of possible decision rules.

The capacity measure later called VC dimension made ‘can this model learn?’ a mathematical question about data and hypothesis classes.

Minsky and Papert rigorously analyzed restricted perceptron architectures and proved important limits on what they could represent.

The book forced researchers to specify architectural assumptions instead of treating ‘learning machines’ as unlimited abstractions.

Linnainmaa's thesis described reverse accumulation of derivatives, while Werbos developed related ordered-derivative methods for nonlinear predictive systems.

These distinct contributions became part of the technical lineage behind later neural-network backpropagation.

The expectation-maximization paper unified a broad alternating procedure for maximum-likelihood estimation with missing or latent data.

Mixtures, missing values, factor models, and many later probabilistic methods gained a reusable estimation pattern.

The Neocognitron alternated S-cells and C-cells in a multilayer hierarchy to learn position-tolerant visual responses.

Its local receptive fields and pooling-like hierarchy made it a direct architectural precursor to convolutional vision systems.

Hopfield showed how symmetric recurrent networks could settle into stable energy minima representing stored patterns.

The network became a content-addressable memory able to reconstruct a full pattern from a corrupted cue.

CART unified classification and regression trees with recursive binary partitioning, evaluation, and pruning.

Trees became a complete statistical modeling system and the base learner for later ensemble methods.

Valiant required a learner to reach bounded error with high confidence using feasible amounts of computation and data.

Learning became a question that joined generalization, sample size, and computational complexity.

Rumelhart, Hinton, and Williams showed that error backpropagation could train multilayer networks whose hidden units discover useful internal representations.

The widely read demonstration renewed neural-network research and made representation learning concrete.

Temporal-difference learning updated a prediction from the difference between successive predictions instead of waiting for a final outcome.

It connected Monte Carlo learning and dynamic programming while supporting incremental, online credit assignment.

Pearl unified probabilistic reasoning through directed belief networks and local message-passing procedures.

Graphical models became a bridge among statistics, AI, diagnosis, causal reasoning, and later machine learning.

Watkins introduced model-free Q-learning. The later paper with Dayan established conditions for convergence to optimal action values.

An agent could learn the value of a target policy while behaving according to another, without a model of the environment.

A network with local connections and shared parameters was trained end to end to recognize handwritten postal digits.

The work moved convolutional ideas from architectural precedent into a practical, backprop-trained vision system.

Soft-margin support-vector machines balanced a wide separating margin against errors and used kernels to work in rich feature spaces.

SVMs became a dependable default for high-dimensional, limited-data classification before deep learning's rise.

AdaBoost repeatedly emphasized difficult examples and combined the resulting weak rules into one weighted classifier.

It turned the theoretical idea of boosting into a practical, influential ensemble algorithm.

LSTM introduced gated memory cells and a persistent error path to learn across long time gaps.

It addressed decaying error signals in recurrent networks and became a durable architecture for language, speech, and time series.

Friedman recast boosting as stage-wise functional gradient descent, allowing additive models to optimize many differentiable losses.

The same functional-gradient view later underpinned widely used boosted-tree implementations.

Random forests combined bootstrap aggregation with random feature selection to build strong, decorrelated tree ensembles.

Random forests paired predictive performance with out-of-bag error estimates and feature-importance measures, usually with modest tuning.

A greedy layer-by-layer procedure made it possible to initialize and train deep generative networks when direct optimization remained difficult.

The layer-wise recipe helped bring deep networks back into active research.

ImageNet organized millions of labeled images through the WordNet hierarchy and supported shared large-scale comparison.

ImageNet gave researchers a common, large-scale test bed for learned visual representations.

The network combined GPUs, ReLUs, augmentation, and dropout to reach a 15.3% top-5 error in the 2012 ImageNet challenge, compared with 26.2% for the runner-up.

That margin helped move computer-vision research toward GPU-trained convolutional networks.

The word2vec architectures made useful distributed word representations inexpensive to train on billion-word corpora.

Embeddings became a practical building block and made relationships in learned vector spaces visible to a broad audience.

Generative adversarial networks trained a generator against a discriminator in a minimax game.

GANs offered an alternative route to sample generation: train a generator through competition with a discriminator.

The translation model learned a soft alignment over source words instead of compressing an entire sentence into one fixed vector.

The learned alignment let the decoder consult different source words at each step, an idea later carried into Transformer architectures.

Residual blocks learned changes relative to an identity path, making substantially deeper networks easier to optimize.

Identity shortcuts became a standard way to train much deeper networks, and the idea later spread beyond image recognition.

AlphaGo combined policy and value networks, supervised learning, reinforcement learning, and Monte Carlo tree search.

The system showed that learned evaluation and tree search could reach championship-level play in a game long viewed as a major AI challenge.

The Transformer performed sequence transduction with attention rather than recurrent or convolutional layers.

Its parallel training and direct token interactions made the Transformer a widely adopted design for later language models and, eventually, multimodal systems.

BERT used masked-token and sentence-level objectives to pretrain deeply bidirectional Transformer representations.

One pretrained model could be fine-tuned across a wide range of language-understanding tasks with minimal task-specific architecture.

GPT-3 scaled autoregressive language modeling to 175 billion parameters and performed many tasks from instructions or examples in its context.

Few-shot and in-context behavior shifted attention from task-specific fine-tuning toward general models prompted at use time.

Denoising diffusion probabilistic models connected diffusion to denoising score matching and produced high-quality samples without adversarial training.

DDPMs showed that iterative denoising could produce competitive samples without adversarial training. Later image systems adapted the same approach to text conditioning.

AlphaFold2 predicted many protein structures with accuracy competitive with experimental structures in the CASP14 assessment.

AlphaFold2 showed that a learned system could reach near-experimental accuracy in a blind protein-structure assessment.

InstructGPT combined demonstrations, preference rankings, reward modeling, and reinforcement learning to improve instruction following.

Human judgments became a scalable training signal for shaping the behavior of general pretrained language models.

GraphCast learned global medium-range weather dynamics and outperformed the operational deterministic HRES system on most reported targets.

On the hardware reported in the paper, GraphCast generated a ten-day forecast in under a minute while matching or beating HRES on most evaluated targets.

AlphaFold 3 extended learned structure prediction from proteins toward complexes involving nucleic acids, small molecules, ions, and modified residues.

AlphaFold 3 brought several kinds of biomolecular complex into one prediction system, including protein-ligand and protein-nucleic-acid interactions.