Alex Krizhevsky
Built AlexNet in 2012 with Ilya Sutskever and Geoffrey Hinton, the deep convolutional network whose ImageNet win is widely credited with kicking off the deep learning boom.
Alex Krizhevsky built AlexNet as a graduate student under Geoffrey HintonGeoffrey HintonKnown as a godfather of deep learning — co-authored backpropagation in 1986, co-invented the Boltzmann machine and dropout, and co-authored AlexNet, the 2012 result that restarted the field. at the University of Toronto, together with Ilya SutskeverIlya SutskeverCo-authored AlexNet as a student, then sequence-to-sequence learning, then co-founded OpenAI and helped drive the bet that scale would produce GPT-level language models.. Entered in the 2012 ImageNet competition, it beat every existing hand-engineered computer vision pipeline by a wide margin — roughly halving the previous best error rate — and is now widely treated as the single result that convinced the rest of the field deep learningDeep LearningDeep learning is machine learning using multi-layer neural networks, which learn their own features from raw data instead of relying on hand-engineered ones. was worth taking seriously again.
The result depended as much on engineering as on the architecture itself: Krizhevsky implemented the training in raw CUDACUDACUDA is NVIDIA's programming platform that lets frameworks like PyTorch dispatch tensor computation to NVIDIA GPUs. and split the network across two GPUsGPU (Graphics Processing Unit)GPUs, originally built for rendering graphics, turned out to be extremely well-suited to the parallel matrix multiplications deep learning requires. to fit it in memory, at a time when training a network this size on GPUs at all was still unusual. That combination — a deep CNNCNN (Convolutional Neural Network)A CNN is a neural network built around the convolution operation, which encodes locality and translation invariance for processing images efficiently., ReLUActivation FunctionAn activation function is the nonlinearity applied after a neuron's weighted sum, without which stacked layers would collapse into one linear function. activations, dropoutOverfittingOverfitting is when a model fits its training data (including noise) so closely that it fails to generalize to new, unseen data., and GPU training done well enough to be practical — became the template computer vision followed for years afterward.
He largely stepped back from AIAI (Artificial Intelligence)AI is the field of building systems that perform tasks normally requiring human intelligence — reasoning, perception, language, and decision-making. research in the years after, but the paper's influence didn't need him to: AlexNetAlexNetAlexNet is the 2012 deep convolutional network that won the ImageNet competition by a wide margin, sparking the deep learning boom. is one of the most-cited papers in computer science, and the "just add more compute and data" lesson it demonstrated echoes through every scaling result that followed it.
See also: Geoffrey HintonGeoffrey HintonKnown as a godfather of deep learning — co-authored backpropagation in 1986, co-invented the Boltzmann machine and dropout, and co-authored AlexNet, the 2012 result that restarted the field., Ilya SutskeverIlya SutskeverCo-authored AlexNet as a student, then sequence-to-sequence learning, then co-founded OpenAI and helped drive the bet that scale would produce GPT-level language models.
Learn more: Computer Vision · History & Landscape
Mentioned in
Lessons where this comes up in context.
Fei-Fei Li
Built ImageNet, the large labeled dataset that made the 2012 deep learning breakthrough in computer vision possible in the first place.
Ian Goodfellow
Invented generative adversarial networks (GANs) in 2014, the architecture that first made realistic image generation possible by pitting two networks against each other.