Technical track

AI Control

This hands-on technical track covers controlling and evaluating AI systems that may intentionally subvert oversight, working through the field's primary literature in full with exercises woven between the sections.

7 modules48 items

Start module 1 Browse the syllabus

No enrollment. Sign in only if you want progress saved.

The modules

Introduction, overview, and threat modeling

1 lesson · 6 readings

The case for ensuring that powerful AIs are controlledreading + exercises105 min

Attack Treeslesson10 min

AI Control: Improving Safety Despite Intentional Subversionreading + exercises + demo94 min

How might we safely pass the buck to AI?reading50 min

Catching AIs red-handedreading + exercises30 min

Prioritizing threats for AI controlreading + exercises15 min

How can we solve diffuse threats like research sabotage with AI control?reading + exercises12 min

Start here~5¾ h

How useful is AI control?

3 lessons · 2 readings

How useful is AI control?lesson

Efficient tradeoffs and the safety-usefulness tradeoff modelreading + exercises

Determining the usefulnesslesson

Plans A, B, C, and D for misalignment riskreading + exercises

Running the modellesson

A Taxonomy of Control

4 lessons · 1 reading

Detection Mitigationslesson18 min

Prevention and Response Mitigationslesson14 min

Control Evaluationslesson8 min

Environmentslesson8 min

An overview of areas of control workreading + exercises40 min

~1½ h

High-stakes control

4 lessons · 10 readings

The high-stakes control roadmap (talk)lesson30 min

What is High Stakes?lesson

Win/continue/lose scenarios and execute/replace/audit protocolsreading + exercises

AI catastrophes and rogue deploymentsreading + exercises

Rogue internal deployments via external APIsreading + exercises

A basic systems architecture for AI agents that do autonomous researchreading + exercises + demo

Monitoring and Mitigationlesson

Practical challenges of control monitoring in frontier AI deploymentsreading

Optional: Blocking live failures with synchronous monitorsreading + exercises + demo

Ctrl-Z: Controlling AI Agents via Resamplingreading + exercises + demo

Environment constructionlesson

Why it's hard to make settings for high-stakes control researchreading + exercises

How to prevent collusion in untrusted monitoring (guided)reading60 min

Basic legibility protocols improve trusted monitoring (guided)reading60 min

Low-stakes control: sabotage, sandbagging, and elicitation

3 lessons · 5 readings

Low-stakes control (ControlConf talk)lesson30 min

Notes on handling non-concentrated failures with AI controllesson26 min

Misalignment and strategic underperformance: an analysis of sandbagging and exploration hackinglesson24 min

Stress-Testing Capability Elicitation (guided)reading + exercises70 min

Exploration Hacking: Can LLMs Learn to Resist RL Training?reading40 min

Auditing Sabotage Bench: A Benchmark for Detecting and Fixing Research Sabotage in ML Codebasesreading35 min

Optional: Stress-Testing Capability Elicitationreading40 min

Exploration Hacking (guided)reading + exercises76 min

~5½ h

Beyond scheming: seekers and deals

3 lessons · 5 readings

Alternatives to Schemerslesson

The case for satiating cheaply-satisfied AI preferencesreading + exercises25 min

Fail safe(r) at alignment by channeling reward-hacking into a "spillway" motivationreading + exercises + demo30 min

Reward Seeker Empiricslesson

Measuring Reward-Seeking via Contrastive Belief Updates (guided)reading75 min

Trading with AIslesson

Making deals with early schemersreading + exercises + demo35 min

A taxonomy of barriers to trading with early misaligned AIsreading + exercises + demo55 min

Next Steps

1 lesson

Next Stepslesson

Prerequisites

AI Safety Fundamentals knowledge

We assume that you have the knowledge equivalent to an intro-level AI safety course (Bluedot, University Intro Courses)

Comfort reading papers

We suggest that you have some experience reading ML or AI Safety research papers & blogposts