David Rumelhart
Cognitive scientist who, with Geoffrey Hinton and Ronald Williams, popularized backpropagation in 1986 — the algorithm that made multi-layer neural networks trainable.
David Rumelhart (1942–2011) was a cognitive psychologist, not an engineer by training, which shaped how he approached neural networksNeural NetworkA neural network is layers of simple weighted-sum-plus-nonlinearity units (neurons) chained together, trained by gradient descent and backpropagation.: as models of how minds might actually process information, not just curve fitters. With Geoffrey HintonGeoffrey HintonKnown as a godfather of deep learning — co-authored backpropagation in 1986, co-invented the Boltzmann machine and dropout, and co-authored AlexNet, the 2012 result that restarted the field. and Ronald Williams, he co-authored the 1986 paper "Learning Representations by Back-propagating Errors" — the paper that made backpropagationBackpropagationBackpropagation is the algorithm that computes the gradient of a neural network's loss with respect to every parameter, by applying the chain rule backward through the network. the standard way to train multi-layer networks.
The algorithm itself had been derived independently by others earlier (including a 1970 formulation by Seppo Linnainmaa and applications by Paul Werbos in the 1970s), but Rumelhart, Hinton, and Williams's paper is what made it land: clear derivation, working experiments, and a direct answer to the objection Marvin MinskyMarvin MinskyCo-founder of the MIT AI Lab and a founding figure of the field — his 1969 book with Seymour Papert exposed the perceptron's limits and helped trigger the first AI winter. and Seymour Papert had raised in 1969 — that single-layer perceptronsPerceptronThe Perceptron (1958) was the first learning system built from an artificial neuron, and the direct ancestor of the neural network training loop. couldn't represent functions like XOR. Backpropagation showed that adding hidden layers, and training all of them jointly by propagating error gradients backward through the network, fixed exactly that limitation.
Rumelhart also co-edited the two-volume Parallel Distributed Processing (1986), the book that gave the resurgent connectionist movement of the 1980s its name and its intellectual foundation.
See also: Geoffrey HintonGeoffrey HintonKnown as a godfather of deep learning — co-authored backpropagation in 1986, co-invented the Boltzmann machine and dropout, and co-authored AlexNet, the 2012 result that restarted the field., Frank RosenblattFrank RosenblattPsychologist who built the perceptron in 1958, the first neural network that learned its own weights from data rather than having them set by hand.
Learn more: Neural Networks & Backprop · Wikipedia: David Rumelhart
Mentioned in
Lessons where this comes up in context.
Frank Rosenblatt
Psychologist who built the perceptron in 1958, the first neural network that learned its own weights from data rather than having them set by hand.
Geoffrey Hinton
Known as a godfather of deep learning — co-authored backpropagation in 1986, co-invented the Boltzmann machine and dropout, and co-authored AlexNet, the 2012 result that restarted the field.