Geoffrey Hinton
Known as a godfather of deep learning — co-authored backpropagation in 1986, co-invented the Boltzmann machine and dropout, and co-authored AlexNet, the 2012 result that restarted the field.
Geoffrey Hinton (b. 1947) spent most of his career as a minority voice insisting neural networksNeural NetworkA neural network is layers of simple weighted-sum-plus-nonlinearity units (neurons) chained together, trained by gradient descent and backpropagation. would eventually work, through two AIAI (Artificial Intelligence)AI is the field of building systems that perform tasks normally requiring human intelligence — reasoning, perception, language, and decision-making. winters that suggested otherwise. With David RumelhartDavid RumelhartCognitive scientist who, with Geoffrey Hinton and Ronald Williams, popularized backpropagation in 1986 — the algorithm that made multi-layer neural networks trainable. and Ronald Williams, he co-authored the 1986 paper that popularized backpropagationBackpropagationBackpropagation is the algorithm that computes the gradient of a neural network's loss with respect to every parameter, by applying the chain rule backward through the network.. Across the following decades he kept building the toolkit deep learningDeep LearningDeep learning is machine learning using multi-layer neural networks, which learn their own features from raw data instead of relying on hand-engineered ones. would eventually run on: Boltzmann machines, restricted Boltzmann machines used for pretrainingPretrainingPretraining is self-supervised training of a base LLM on massive amounts of text to predict the next token, the primary source of its knowledge and language ability. before better initialization made that unnecessary, and dropout — randomly disabling neurons during training, still one of the most widely used regularizationRegularizationRegularization is any technique that trades some training-data fit for better generalization, fighting overfitting on purpose. techniques.
The result that changed everything came in 2012. Hinton, with students Ilya SutskeverIlya SutskeverCo-authored AlexNet as a student, then sequence-to-sequence learning, then co-founded OpenAI and helped drive the bet that scale would produce GPT-level language models. and Alex KrizhevskyAlex KrizhevskyBuilt AlexNet in 2012 with Ilya Sutskever and Geoffrey Hinton, the deep convolutional network whose ImageNet win is widely credited with kicking off the deep learning boom., built AlexNetAlexNetAlexNet is the 2012 deep convolutional network that won the ImageNet competition by a wide margin, sparking the deep learning boom., a CNNCNN (Convolutional Neural Network)A CNN is a neural network built around the convolution operation, which encodes locality and translation invariance for processing images efficiently. that won the ImageNet competition by a margin so large — roughly halving the error rate of the next-best entry — that it ended a long-running argument about whether deep networks could outperform hand-engineered computer vision features. AlexNet is generally treated as the moment deep learning went from a promising niche to the dominant paradigm in machine learningMachine Learning (ML)Machine learning is the practice of writing programs that learn a function from data, via a loss function and an optimizer, instead of following hand-written rules..
Hinton, Yann LeCunYann LeCunInvented the convolutional neural network in the late 1980s and deployed it commercially reading handwritten checks years before deep learning was fashionable., and Yoshua BengioYoshua BengioHelped lay the statistical foundations of modern language modeling and co-founded the Montreal deep learning school — now a leading voice on AI safety. shared the 2018 Turing Award — computing's equivalent of the Nobel Prize — for their contributions to deep learning, and the trio is commonly called the "godfathers of deep learning." In 2023, Hinton left Google specifically to speak more freely about AI safetyAI SafetyAI safety is the question of whether a system's own behavior matches what its designers actually want, independent of any attacker. risks he believes the field is not taking seriously enough.
See also: David RumelhartDavid RumelhartCognitive scientist who, with Geoffrey Hinton and Ronald Williams, popularized backpropagation in 1986 — the algorithm that made multi-layer neural networks trainable., Yann LeCunYann LeCunInvented the convolutional neural network in the late 1980s and deployed it commercially reading handwritten checks years before deep learning was fashionable., Yoshua BengioYoshua BengioHelped lay the statistical foundations of modern language modeling and co-founded the Montreal deep learning school — now a leading voice on AI safety., Ilya SutskeverIlya SutskeverCo-authored AlexNet as a student, then sequence-to-sequence learning, then co-founded OpenAI and helped drive the bet that scale would produce GPT-level language models.
Learn more: Neural Networks & Backprop · Computer Vision · Wikipedia: Geoffrey Hinton
Mentioned in
Lessons where this comes up in context.
David Rumelhart
Cognitive scientist who, with Geoffrey Hinton and Ronald Williams, popularized backpropagation in 1986 — the algorithm that made multi-layer neural networks trainable.
Yann LeCun
Invented the convolutional neural network in the late 1980s and deployed it commercially reading handwritten checks years before deep learning was fashionable.