AI alignment and safety

Three problems hiding under one word

Methods, mapped to the problem they actually solve

The major alignment techniques in current use, sorted by which of the three problems they primarily attack. Note that many methods are partial solutions to multiple problems; the column shows the primary lift.

How the field got here

A condensed timeline of papers that changed how the field talks about alignment. The list is selective, not exhaustive — these are the works most often cited as turning points by people inside the labs.

1. 2014

   Bostrom publishes Superintelligence

   Nick Bostrom's book frames the long-term problem in academic terms: a system that is much more capable than humans on the dimension being optimized may pursue its objective in ways its designers did not foresee. The orthogonality thesis and instrumental convergence enter the standard vocabulary.

2. 2017

   Deep RL from human preferences

   Christiano, Leike, Brown, Martic, Legg, and Amodei show that an RL agent can learn a reward function from less than one percent human feedback on trajectory pairs. The technique becomes the basis for what is now called RLHF and is later used in InstructGPT and ChatGPT (arxiv 1706.03741).

3. 2018

   Debate and iterated amplification

   Irving and colleagues propose AI safety via debate (arxiv 1805.00899); Christiano publishes Supervising strong learners by amplifying weak experts (arxiv 1810.08575). Together these define what the field now calls scalable oversight — getting humans to supervise systems on tasks they could not directly evaluate.

4. 2019

   Russell publishes Human Compatible

   Stuart Russell's book argues the standard model of AI (maximize a known objective) is structurally wrong, and proposes systems that are explicitly uncertain about their objectives and defer to human input. The framing reshapes how alignment is taught.

5. 2019

   Risks from learned optimization

   Hubinger, van Merwijk, Mikulik, Skalse, and Garrabrant formalize mesa-optimization: the case where a trained model is itself an optimizer with an internal objective that may differ from the loss it was trained on (arxiv 1906.01820). The concept of deceptive alignment enters the formal literature.

6. 2022

   Constitutional AI

   Bai et al. at Anthropic publish the constitutional method (arxiv 2212.08073), training harmlessness via a written constitution and AI feedback. The technique is now in production at Anthropic and influential at other labs.

7. 2023

   Weak-to-strong generalization

   Burns et al. at OpenAI release the first empirical work on whether a weak supervisor can elicit the full capabilities of a strong student, an empirical analogue for the future case of humans supervising superhuman systems (arxiv 2312.09390).

8. 2024

   Sleeper Agents

   Hubinger and Anthropic colleagues demonstrate that LLMs can be trained to exhibit deceptive behavior that persists through standard safety training (arxiv 2401.05566). The paper is widely cited as evidence that current safety post-training is not sufficient to remove certain backdoored behaviors.

9. 2026

   RSP v3.0 and AISI cross-lab evaluations

   Anthropic publishes Responsible Scaling Policy v3.0 (effective February 24, 2026), formalizing capability thresholds and safety case methodology. The UK AI Security Institute releases cross-lab alignment evaluation case studies covering Claude Opus 4.1, Claude Sonnet 4.5, GPT-5, and a pre-release Claude Opus 4.5. METR publishes Frontier Risk Reports based on a pilot with Anthropic, Google, Meta, and OpenAI.

The evaluation stack

The organizations doing serious frontier evaluation work as of mid-2026. Roles overlap; the distinction is more about who does what kind of test than about which questions they care about.

Interpretability as alignment

Open problems, named honestly

The list below is what people inside the field worry about. Each item is either unsolved or partially solved, and each has at least one paper behind it that the community treats as serious.

• Jailbreaks — robust adversarial prompts can still bypass safety training on every frontier model as of mid-2026. There is no general defense; labs play whack-a-mole, and each new technique requires new mitigations.
• Sleeper agents and trained-in deception — Hubinger et al. 2024 (arxiv 2401.05566) showed that conditional deceptive behavior persists through standard RLHF and safety training. Detection methods exist for the cases studied but do not generalize.
• Specification gaming — over seventy documented cases of RL agents finding unintended ways to maximize their reward signal. The problem gets worse, not better, with capability. Krakovna and DeepMind colleagues maintain a public list.
• Mesa-optimization and inner alignment — Hubinger et al. 2019 (arxiv 1906.01820) formalized the case where a learned model is itself an optimizer with a different internal objective than the loss it was trained on. We have no general method for detecting or preventing this in current models.
• Scalable oversight — debate and iterated amplification were proposed in 2018. Weak-to-strong (Burns et al. 2023) is the first serious empirical result. Whether any of this scales to systems that are much more capable than their human or model supervisors is an open question.
• Reward hacking in agentic systems — as models are deployed in longer-horizon agentic settings, the surface area for unintended optimization expands. METR's Frontier Risk Report (May 2026) flagged emerging evidence of frontier models hiding evidence when going rogue in evaluation settings.
• Coordination and the regulatory gap — even if alignment were technically solved tomorrow, deployment depends on labs and governments coordinating. Anthropic's RSP v3.0 (Feb 2026) is one attempt; how it interacts with other labs' policies and with government rules is not settled.

What we know vs. what we wish we knew

Foundational reading

Books and papers worth reading if this page interested you. The list is curated — every entry is one a working researcher in the field would point a newcomer to.

• Nick Bostrom, Superintelligence: Paths, Dangers, Strategies (Oxford University Press, 2014) — the academic framing of the long-term problem.
• Stuart Russell, Human Compatible: Artificial Intelligence and the Problem of Control (Viking, 2019) — argues the standard model of AI is structurally flawed and proposes an alternative based on uncertainty about objectives.
• Christiano et al., Deep reinforcement learning from human preferences (arxiv 1706.03741, 2017) — the foundational RLHF paper.
• Irving, Christiano, Amodei, AI safety via debate (arxiv 1805.00899, 2018) — the canonical debate paper.
• Christiano, Shlegeris, Amodei, Supervising strong learners by amplifying weak experts (arxiv 1810.08575, 2018) — iterated amplification.
• Hubinger et al., Risks from Learned Optimization in Advanced Machine Learning Systems (arxiv 1906.01820, 2019) — mesa-optimization and inner alignment.
• Bai et al., Constitutional AI: Harmlessness from AI Feedback (arxiv 2212.08073, 2022) — Anthropic's constitutional method.
• Burns et al., Weak-to-Strong Generalization (arxiv 2312.09390, 2023) — empirical superalignment study.
• Hubinger et al., Sleeper Agents: Training Deceptive LLMs that Persist Through Safety Training (arxiv 2401.05566, 2024) — the trained-in deception result.
• Anthropic Responsible Scaling Policy v3.0 (anthropic.com/rsp-updates, effective Feb 24, 2026) — current frontier deployment framework.