Ashish Vaswani
Lead author of "Attention Is All You Need" (2017), the paper that introduced the transformer architecture behind every modern LLM.
Ashish Vaswani led the eight-author team at Google that published "Attention Is All You Need" in 2017 — the paper that introduced the transformerTransformerThe transformer is the neural network architecture built around self-attention, introduced in 2017, underlying essentially all modern LLMs.. Its central claim was that attentionAttention (Self-Attention)Attention is a mechanism letting each position in a sequence weigh every other position via learned Query/Key/Value vectors, forming the core of the transformer. alone, without the recurrence used by RNNsRNN (Recurrent Neural Network)An RNN is a neural network for sequences that processes input one step at a time, carrying a hidden state forward — the main predecessor to transformers. or the local receptive fields used by CNNsCNN (Convolutional Neural Network)A CNN is a neural network built around the convolution operation, which encodes locality and translation invariance for processing images efficiently., was sufficient to model sequences, and that removing recurrence's inherently sequential computation meant every position in a sequence could be processed in parallel during training. That single architectural change is why training runs that would have taken months on an RNN became feasible in days on a transformer, and it's the reason every major LLMLLM (Large Language Model)An LLM is a large transformer trained to predict the next token on massive text corpora, then fine-tuned to follow instructions — the architecture behind GPT, Claude, Gemini, and Llama. built since — GPT, BERT, Claude, and the rest — shares the same underlying architecture.
The paper's co-authors — Noam ShazeerNoam ShazeerCo-authored "Attention Is All You Need" and pioneered sparse mixture-of-experts models, then left Google to found Character.AI before returning to lead work on Gemini., Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Łukasz Kaiser, and Illia Polosukhin — have themselves gone on to found or shape several major AIAI (Artificial Intelligence)AI is the field of building systems that perform tasks normally requiring human intelligence — reasoning, perception, language, and decision-making. labs and products, making "Attention Is All You Need" one of the most consequential single papers in the field's history, cited well over 100,000 times.
Vaswani later co-founded Adept AI and Essential AI, both focused on applying transformer-based models to autonomous agentsAgentAn agent puts an LLM in a loop with tool access, letting it decide autonomously which tools to call and in what order to accomplish a multi-step goal. and enterprise tasks — a natural extension of an architecture originally designed for machine translation.
See also: Ilya SutskeverIlya SutskeverCo-authored AlexNet as a student, then sequence-to-sequence learning, then co-founded OpenAI and helped drive the bet that scale would produce GPT-level language models., Noam ShazeerNoam ShazeerCo-authored "Attention Is All You Need" and pioneered sparse mixture-of-experts models, then left Google to found Character.AI before returning to lead work on Gemini.
Learn more: Attention & Transformers · Wikipedia: Attention Is All You Need
Ilya Sutskever
Co-authored AlexNet as a student, then sequence-to-sequence learning, then co-founded OpenAI and helped drive the bet that scale would produce GPT-level language models.
Noam Shazeer
Co-authored "Attention Is All You Need" and pioneered sparse mixture-of-experts models, then left Google to found Character.AI before returning to lead work on Gemini.