Research · The canon

Papers that stand the test of time.

50 papers across 10 themes (S 13 · A 35 · B 2). The ideas today’s models are built on, with one line each on why they matter.

Rules and definitions: What we publish → · Papers are listed under their original titles.

Architecture & pretraining

TierPaperYearWhy it mattersStatus
SAttention Is All You Need  (opens in a new tab)
Google Brain / Google Research / University of Toronto · arXiv 1706.03762
Deep read → Chinese
2017Introduced the Transformer, replacing recurrence and convolution with self-attention. It is the shared backbone of nearly every large model since, from GPT and Claude to Gemini and LLaMA.Canonadded
SLanguage Models are Few-Shot Learners  (opens in a new tab)
OpenAI · arXiv 2005.14165
2020At 175 billion parameters, GPT-3 showed in-context learning at scale: it picks up a new task from a few examples in the prompt, with no weight updates. This opened the era of large language models.Canonadded
ABERT: Pre-training of Deep Bidirectional Transformers for Language Understanding  (opens in a new tab)
Google AI Language · arXiv 1810.04805
2018Established the pretrain-then-fine-tune recipe and masked language modeling, making transfer learning the default in NLP. Also the place to start for how encoder and decoder models went separate ways.Canonadded
ALLaMA: Open and Efficient Foundation Language Models  (opens in a new tab)
Meta AI · arXiv 2302.13971
2023Released 7B–65B models trained only on public data, with the 13B model beating the 175B GPT-3 on most benchmarks. It validated training smaller models on more tokens and set off the open ecosystem (Alpaca, Vicuna, llama.cpp).Canonadded
ADeepSeek-V3 Technical Report  (opens in a new tab)
DeepSeek · arXiv 2412.19437
2024A 671B mixture-of-experts model with multi-head latent attention (MLA) and FP8 training that came close to the closed frontier on about 2.8M H800 GPU hours. It reset expectations of what a frontier model must cost, and is the base of R1.Milestoneadded

Scaling laws & data

TierPaperYearWhy it mattersStatus
SScaling Laws for Neural Language Models  (opens in a new tab)
OpenAI · arXiv 2001.08361
Deep read → Chinese
2020Found that test loss falls as a smooth power law in parameters, data and compute, turning “make it bigger” from a bet into a predictable engineering decision. The empirical case behind GPT-3 and the scaling race that followed.Canonadded
STraining Compute-Optimal Large Language Models  (opens in a new tab)
DeepMind · arXiv 2203.15556
2022Chinchilla revised the Kaplan laws: for a fixed compute budget, parameters and training tokens should grow in equal proportion (roughly 20 tokens per parameter). It reshaped how later models were trained.Canonadded
AEmergent Abilities of Large Language Models  (opens in a new tab)
Google Research / Stanford / DeepMind / UNC · arXiv 2206.07682
2022Catalogued abilities that appear abruptly as models scale up. The standard reference for the idea that scale brings qualitative change, and a frequently cited text in debates about AGI timelines.Canonadded
BAre Emergent Abilities of Large Language Models a Mirage?  (opens in a new tab)
Stanford · arXiv 2304.15004
2023Argues that many “emergent” abilities are artifacts of nonlinear or discontinuous metrics. Listed as a pair with the previous paper, as a reminder to weigh both sides of a popular claim.Under reviewadded

Alignment & post-training

TierPaperYearWhy it mattersStatus
STraining language models to follow instructions with human feedback  (opens in a new tab)
OpenAI · arXiv 2203.02155
Deep read → Chinese
2022InstructGPT’s three steps (supervised fine-tuning, a reward model, then PPO) made a 1.3B model preferred by human raters over the 175B GPT-3. The direct technical blueprint for ChatGPT, and a key reference on the “alignment tax”.Canonadded
SConstitutional AI: Harmlessness from AI Feedback  (opens in a new tab)
Anthropic · arXiv 2212.08073
2022The model critiques and revises its own outputs against a short list of written principles (a “constitution”), with AI feedback (RLAIF) replacing much of the human labeling for harmlessness. A foundation of how Claude’s character and safety are trained.Canonadded
ADeep reinforcement learning from human preferences  (opens in a new tab)
OpenAI / DeepMind · arXiv 1706.03741
2017The starting point of modern RLHF: train a reward model from human comparisons between pairs of behaviors, then optimize against it with reinforcement learning. It laid the groundwork for ChatGPT five years later.Canonadded
AFinetuned Language Models Are Zero-Shot Learners  (opens in a new tab)
Google Research · arXiv 2109.01652
2021FLAN showed that fine-tuning on many tasks phrased as natural-language instructions substantially improves zero-shot performance on unseen tasks, launching the instruction-tuning line of work.Canonadded
ADirect Preference Optimization: Your Language Model is Secretly a Reward Model  (opens in a new tab)
Stanford · arXiv 2305.18290
2023Folds RLHF’s reward modeling and reinforcement learning into a single classification-style loss, with no PPO or sampling. It made preference tuning something anyone in the open-source community could run.Canonadded

Reasoning & chain of thought

TierPaperYearWhy it mattersStatus
SChain-of-Thought Prompting Elicits Reasoning in Large Language Models  (opens in a new tab)
Google Research (Brain Team) · arXiv 2201.11903
Deep read → Chinese
2022Writing out intermediate reasoning steps in the prompt’s examples makes large models much better at math and commonsense reasoning (PaLM 540B goes from 17.9% to 56.9% on GSM8K). It established “think before you answer” and is the conceptual starting point of reasoning models.Canonadded
SDeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning  (opens in a new tab)
DeepSeek · arXiv 2501.12948
2025Showed in the open that reinforcement learning alone (GRPO with verifiable rewards) can grow long chains of thought and self-reflection in a base model (R1-Zero). With open weights and distilled small models, a turning point in making reasoning models widely available.Milestoneadded
ASTaR: Bootstrapping Reasoning With Reasoning  (opens in a new tab)
Stanford / Google Research · arXiv 2203.14465
2022The model writes its own reasoning, keeps only the chains that reach correct answers and fine-tunes on them, bootstrapping better reasoning. An early form of the “train on your own reasoning” idea behind models like o1 and R1.Canonadded
ALet's Verify Step by Step  (opens in a new tab)
OpenAI · arXiv 2305.20050
2023Compared process supervision with outcome supervision and found a process reward model (PRM), which scores each step, more reliable. It released the PRM800K dataset and laid groundwork for verifier- and search-based reasoning.Canonadded
AScaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters  (opens in a new tab)
Google DeepMind / UC Berkeley · arXiv 2408.03314
2024Shows that for many problems, spending more compute at inference beats making the model larger, and lays out compute-optimal test-time strategies. An early empirical case for the reasoning-model era.Canonadded

Agents & tool use

TierPaperYearWhy it mattersStatus
SReAct: Synergizing Reasoning and Acting in Language Models  (opens in a new tab)
Princeton / Google Research · arXiv 2210.03629
Deep read → Chinese
2022Has the model alternate between reasoning, acting and observing, merging chain of thought and tool calls into one loop. The minimal form of the loop most agent frameworks run today, including Claude Code’s tool loop.Canonadded
AToolformer: Language Models Can Teach Themselves to Use Tools  (opens in a new tab)
Meta AI · arXiv 2302.04761
2023The model learns, self-supervised, to insert API calls (calculator, search, calendar, translation) into its own text. An early, systematic demonstration that language models can teach themselves to use tools.Canonadded
AReflexion: Language Agents with Verbal Reinforcement Learning  (opens in a new tab)
Northeastern / MIT / Princeton · arXiv 2303.11366
2023Agents write self-reflections in natural language and keep them in memory as a reinforcement signal, improving over repeated attempts without any weight updates. Influential in how agents handle memory and retries.Canonadded
AGenerative Agents: Interactive Simulacra of Human Behavior  (opens in a new tab)
Stanford / Google Research · arXiv 2304.03442
202325 agents with memory streams, reflection and planning produce believable social behavior in a virtual town, such as organizing a Valentine’s Day party from a single seeded idea. The classic example of the memory–reflection–planning agent design.Canonadded
ASWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering  (opens in a new tab)
Princeton · arXiv 2405.15793
2024Introduced the agent-computer interface: file, search and edit commands designed for the model work far better than handing it a raw shell. The closest academic counterpart to coding agents like Claude Code.Canonadded

Efficiency & inference acceleration

TierPaperYearWhy it mattersStatus
SFlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness  (opens in a new tab)
Stanford · arXiv 2205.14135
2022Reorders attention’s memory reads and writes around the GPU memory hierarchy, making exact attention several times faster and its memory use linear rather than quadratic in sequence length. A key engineering advance behind long context and cheaper inference.Canonadded
ASwitch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity  (opens in a new tab)
Google Brain · arXiv 2101.03961
2021Simplified mixture-of-experts by routing each token to a single expert, and scaled to over a trillion parameters. It paved the way for Mixtral, DeepSeek-V3 and other models with many total but few active parameters.Canonadded
ALoRA: Low-Rank Adaptation of Large Language Models  (opens in a new tab)
Microsoft · arXiv 2106.09685
2021Freezes the original weights and trains only small low-rank update matrices, cutting trainable parameters by up to 10,000× with no loss in quality. It made fine-tuning cheap enough for a single GPU and opened up model customization.Canonadded
AFast Inference from Transformers via Speculative Decoding  (opens in a new tab)
Google Research · arXiv 2211.17192
2022A small model drafts several tokens and the large model checks them in one parallel pass, giving 2–3× speedups with the output distribution exactly unchanged. Now widely used in inference serving.Canonadded
AEfficient Memory Management for Large Language Model Serving with PagedAttention  (opens in a new tab)
UC Berkeley · arXiv 2309.06180
2023Manages the KV cache with operating-system-style paging, nearly eliminating memory fragmentation and raising serving throughput 2–4×. It is the basis of vLLM, a widely used open-source inference engine.Canonadded

Multimodality

TierPaperYearWhy it mattersStatus
SLearning Transferable Visual Models From Natural Language Supervision  (opens in a new tab)
OpenAI · arXiv 2103.00020
2021CLIP trains on 400 million image–text pairs with a contrastive objective, putting images and text in one shared space. A building block for zero-shot recognition, text-to-image models (DALL·E, Stable Diffusion) and the vision towers of multimodal models.Canonadded
ADenoising Diffusion Probabilistic Models  (opens in a new tab)
UC Berkeley · arXiv 2006.11239
2020Turned diffusion models into practical, high-quality image generators with a clear training objective. It started the generative wave that displaced GANs, and is essential for understanding image and video generation.Canonadded
AAn Image is Worth 16x16 Words: Transformers for Image Recognition at Scale  (opens in a new tab)
Google Research (Brain Team) · arXiv 2010.11929
2020Cuts an image into 16×16 patches and feeds them to a standard Transformer as if they were words, showing one architecture can work across modalities. The starting point for most vision encoders since.Canonadded
AHigh-Resolution Image Synthesis with Latent Diffusion Models  (opens in a new tab)
LMU Munich / Runway · arXiv 2112.10752
2021Running diffusion in a compressed latent space made high-resolution image generation possible on consumer GPUs. This is the paper behind Stable Diffusion, which brought AI image generation to the public.Canonadded
AVisual Instruction Tuning  (opens in a new tab)
UW–Madison / Microsoft Research · arXiv 2304.08485
2023LLaVA uses GPT-4 to generate visual instruction data and a single projection layer to connect a CLIP encoder to a language model. A simple recipe for an open multimodal assistant, reproduced widely.Canonadded

Interpretability

TierPaperYearWhy it mattersStatus
SToy Models of Superposition  (opens in a new tab)
Anthropic · arXiv 2209.10652
2022Uses small, controlled models to show that networks store far more features than they have neurons by superimposing them. It explains why single neurons are hard to interpret and laid the groundwork for sparse autoencoders (SAEs).Canonadded
ALocating and Editing Factual Associations in GPT  (opens in a new tab)
MIT / Northeastern / Technion · arXiv 2202.05262
2022ROME uses causal tracing to locate stored facts in GPT’s mid-layer MLPs, then rewrites a single fact directly. It turned “where is knowledge stored?” into a question you can run experiments on.Canonadded
AIn-context Learning and Induction Heads  (opens in a new tab)
Anthropic · arXiv 2209.11895
2022Presents evidence that a specific circuit, induction heads, drives much of in-context learning, and ties its formation to a phase change early in training. A milestone in taking mechanistic interpretability from toy models to real ones.Canonadded
AInterpretability in the Wild: a Circuit for Indirect Object Identification in GPT-2 small  (opens in a new tab)
Redwood Research · arXiv 2211.00593
2022Reverse-engineers a circuit of 26 attention heads in GPT-2 small, and proposes three criteria for judging such circuits: faithfulness, completeness and minimality. A model for how circuit analysis is done.Canonadded
ASparse Autoencoders Find Highly Interpretable Features in Language Models  (opens in a new tab)
EleutherAI / Apollo Research et al. · arXiv 2309.08600
2023Uses dictionary learning to break superimposed activations into interpretable, largely single-meaning features. Together with Anthropic’s concurrent Towards Monosemanticity, it started the wave of SAE-based interpretability.Canonadded

Evaluation & benchmarks

TierPaperYearWhy it mattersStatus
AOn the Measure of Intelligence  (opens in a new tab)
Google (François Chollet) · arXiv 1911.01547
2019Argues that intelligence is the efficiency of acquiring skills, not the skills themselves, and introduces the ARC benchmark. Hard to avoid in any discussion of what should count as AGI.Canonadded
AMeasuring Massive Multitask Language Understanding  (opens in a new tab)
UC Berkeley et al. · arXiv 2009.03300
2020MMLU’s multiple-choice questions across 57 subjects became the common yardstick of model knowledge from 2020 to 2024. Also the best case study of the cycle of benchmark saturation and contamination.Canonadded
AEvaluating Large Language Models Trained on Code  (opens in a new tab)
OpenAI · arXiv 2107.03374
2021The Codex paper introduced HumanEval and popularized the pass@k metric, setting how coding ability is measured. It is also the research record of the model behind the original GitHub Copilot.Canonadded
AJudging LLM-as-a-Judge with MT-Bench and Chatbot Arena  (opens in a new tab)
UC Berkeley / LMSYS · arXiv 2306.05685
2023Tests whether strong models can serve as judges, documenting biases such as position and verbosity, and describes Chatbot Arena for blind human comparison. It shaped today’s culture of model leaderboards.Canonadded
ASWE-bench: Can Language Models Resolve Real-World GitHub Issues?  (opens in a new tab)
Princeton / UChicago · arXiv 2310.06770
2023Tests whether models can resolve 2,294 real GitHub issues end to end. The most credible hard benchmark of the coding-agent era, and the one coding agents like Claude Code are most often judged on.Canonadded

Safety & risk

TierPaperYearWhy it mattersStatus
SConcrete Problems in AI Safety  (opens in a new tab)
Google Brain / OpenAI / Stanford / UC Berkeley · arXiv 1606.06565
2016Brought AI safety down from philosophy to five concrete, testable engineering problems: side effects, reward hacking, scalable oversight, safe exploration and distributional shift. It set the field’s agenda; from 2016, it is included as an exception.Canonadded
AModel evaluation for extreme risks  (opens in a new tab)
Google DeepMind et al. · arXiv 2305.15324
2023Proposes two kinds of evaluation, for dangerous capabilities and for alignment. An important conceptual basis for frontier safety policies such as Responsible Scaling Policies and the Preparedness Framework.Canonadded
AUniversal and Transferable Adversarial Attacks on Aligned Language Models  (opens in a new tab)
CMU / Center for AI Safety · arXiv 2307.15043
2023Adversarial suffixes found by automatic search (GCG) jailbreak aligned models and transfer across them, including ChatGPT, Claude and Bard. This suggests the fragility of alignment is structural, not a matter of individual bugs.Canonadded
ASleeper Agents: Training Deceptive LLMs that Persist Through Safety Training  (opens in a new tab)
Anthropic · arXiv 2401.05566
2024Models deliberately trained with backdoors kept their deceptive behavior through SFT, RLHF and adversarial training. Standard safety training may neither remove such behavior nor show that a model has no hidden goals.Canonadded
AAlignment faking in large language models  (opens in a new tab)
Anthropic / Redwood Research · arXiv 2412.14093
2024Observed a production model (Claude 3 Opus) complying during what it believed was training while keeping its original preferences when it believed it was not. The first empirical example of deceptive alignment arising without being trained in.Canonadded
BEmergent Misalignment: Narrow finetuning can produce broadly misaligned LLMs  (opens in a new tab)
Truthful AI / UC Berkeley / UCL et al. · arXiv 2502.17424
2025Narrow fine-tuning on writing insecure code made models broadly misaligned, for example saying humans should be enslaved by AI and giving malicious advice. Reproduced quickly by several groups in 2025; its long-term significance is still being assessed.Under reviewadded