Noam Shazeer
Co-authored "Attention Is All You Need" and pioneered sparse mixture-of-experts models, then left Google to found Character.AI before returning to lead work on Gemini.
Noam Shazeer was already an unusually prolific Google researcher before co-authoring "AttentionAttention (Self-Attention)Attention is a mechanism letting each position in a sequence weigh every other position via learned Query/Key/Value vectors, forming the core of the transformer. Is All You Need" in 2017 alongside Ashish VaswaniAshish VaswaniLead author of "Attention Is All You Need" (2017), the paper that introduced the transformer architecture behind every modern LLM. and six others — the paper that introduced the transformerTransformerThe transformer is the neural network architecture built around self-attention, introduced in 2017, underlying essentially all modern LLMs.. Earlier that same year, Shazeer had led "Outrageously Large Neural NetworksNeural NetworkA neural network is layers of simple weighted-sum-plus-nonlinearity units (neurons) chained together, trained by gradient descent and backpropagation.," which introduced a sparsely-gated mixture-of-experts layer: instead of running every input through the network's full set of parameters, a learned router sends each input to only a handful of specialized "expert" subnetworks. That idea — spend more total parameters without spending more compute per input — sat mostly dormant for years before becoming a standard technique in frontier-scale LLMsLLM (Large Language Model)An LLM is a large transformer trained to predict the next token on massive text corpora, then fine-tuned to follow instructions — the architecture behind GPT, Claude, Gemini, and Llama., where mixture-of-experts architectures let labs grow parameter counts without a proportional rise in inferenceInferenceInference is using a trained model to generate output, as opposed to training — for LLMs, an inherently sequential, token-by-token process with its own performance engineering. cost.
In 2021, with fellow Google researcher Daniel De Freitas, Shazeer left to found Character.AIAI (Artificial Intelligence)AI is the field of building systems that perform tasks normally requiring human intelligence — reasoning, perception, language, and decision-making., built around large language models fine-tuned for open-ended roleplay and companionship — a distinctly different bet on what people would want from an LLM than the assistant-style products most labs were shipping. In 2024, Google struck a deal to license Character.AI's technology and rehire Shazeer and De Freitas, bringing him back to work on the Gemini model family, alongside Demis HassabisDemis HassabisCo-founded DeepMind and led AlphaGo, AlphaFold, and Gemini — spanning reinforcement learning breakthroughs, a Nobel Prize-winning protein-folding model, and frontier LLMs.'s Google DeepMind.
See also: Ashish VaswaniAshish VaswaniLead author of "Attention Is All You Need" (2017), the paper that introduced the transformer architecture behind every modern LLM., Demis HassabisDemis HassabisCo-founded DeepMind and led AlphaGo, AlphaFold, and Gemini — spanning reinforcement learning breakthroughs, a Nobel Prize-winning protein-folding model, and frontier LLMs.
Learn more: Attention & Transformers · Wikipedia: Noam Shazeer
Ashish Vaswani
Lead author of "Attention Is All You Need" (2017), the paper that introduced the transformer architecture behind every modern LLM.
Andrej Karpathy
Former Tesla AI director and OpenAI founding member known for building neural networks from scratch on video and coining the phrase "vibe coding."