Paper

Basic legibility protocols improve trusted monitoring (guided)

Ashwin Sreevatsa, Sebastian Prasanna, Cody Rushing

Abstract

The AI Control research agenda aims to develop control protocols: safety techniques that prevent untrusted AI systems from taking harmful actions during deployment. Because human oversight is expensive, one approach is trusted monitoring, where weaker, trusted models oversee stronger, untrusted models—but this often fails when the untrusted model’s actions exceed the monitor’s comprehension. We introduce legibility protocols, which encourage the untrusted model to take actions that are easier for a monitor to evaluate.

2 Introduction

AI systems have rapidly improved across a wide range of difficult, economically valuable tasks [1, 2, 3]. AI companies may soon deploy these systems internally to automate security-critical work such as software engineering and AI research and development [4]. As these systems take on less well-defined tasks, a key challenge will be ensuring appropriate oversight to prevent such systems from taking harmful actions, especially if they are misaligned and actively attempting to subvert human goals. Such systems may be able to avoid having their misalignment detected by ”faking alignment” during testing [5].

AI Control aims to develop deployment-time safety measures robust to intentional subversion by AI agents. Prior work introduced control evaluations as an adversarial framework for assessing control protocols based on their ability to complete assigned tasks (usefulness) while preventing harmful actions (safety) [6]. These evaluations pit a blue team (developers) against a red team (simulating a misaligned model). The blue team designs protocols using a strong but untrusted model, a weaker trusted model, and limited human auditing, while the red team attempts to subvert the protocol by having the untrusted model covertly take malicious actions.

One class of protocols previously studied is trusted monitoring, where the weaker, trusted model monitors the outputs of the stronger, untrusted model and flags suspicious outputs for human auditing. However, a core challenge is that the weaker monitor often cannot effectively oversee the stronger model because it does not fully understand the latter’s outputs. To address this, we introduce legibility protocols, which augment trusted monitoring by encouraging the untrusted model to take actions the monitor can more easily evaluate.

Before you read on

Propose 2-3 ways to make U's code easier for T to understand and therefore evaluate. For each method, write about how it increases the legibility of U's logic to T.

120 more characters to go

References

Tejal Patwardhan, Rachel Dias, Elizabeth Proehl, Grace Kim, Michele Wang, Olivia Watkins, Simon Posada Fishman, et al. GDPval: Evaluating AI Model Performance on Real-World Economically Valuable Tasks. arXiv preprint arXiv:2510.04374. 2025.
↩
Thomas Kwa, Ben West, Joel Becker, Amy Deng, Katharyn Garcia, Max Hasin, Sami Jawhar, et al. Measuring AI Ability to Complete Long Tasks. arXiv preprint arXiv:2503.14499. 2025.
Axel Backlund and Lukas Petersson. Vending-Bench: A Benchmark for Long-Term Coherence of Autonomous Agents. arXiv preprint arXiv:2502.15840. 2025.
Hjalmar Wijk, et al. RE-Bench: Evaluating Frontier AI R&D Capabilities of Language Model Agents Against Human Experts. arXiv preprint arXiv:2411.15114. 2025.
↩
Joe Carlsmith. Scheming AIs: Will AIs Fake Alignment During Training in Order to Get Power?. arXiv preprint arXiv:2311.08379. 2023.
↩
Ryan Greenblatt, Buck Shlegeris, Kshitij Sachan and Fabien Roger. AI Control: Improving Safety Despite Intentional Subversion. In Proceedings of the 41st International Conference on Machine Learning, pages 16295–16336. 2024.
↩
Dan Hendrycks, Steven Basart, Saurav Kadavath, Mantas Mazeika, Akul Arora, Ethan Guo, Collin Burns, et al. Measuring Coding Challenge Competence with APPS. arXiv preprint arXiv:2105.09938. 2021.
↩
Aryan Bhatt, Cody Rushing, Adam Kaufman, Tyler Tracy, Vasil Georgiev, David Matolcsi, Akbir Khan and Buck Shlegeris. Ctrl-Z: Controlling AI Agents via Resampling. arXiv preprint arXiv:2504.10374. 2025.
↩
Jonathan Kutasov, Yuqi Sun, Paul Colognese, Teun van der Weij, Linda Petrini, Chen Bo Calvin Zhang, John Hughes, et al. SHADE-Arena: Evaluating Sabotage and Monitoring in LLM Agents. arXiv preprint arXiv:2506.15740. 2025.
↩
Adam Kaufman, James Lucassen, Tyler Tracy, Cody Rushing and Aryan Bhatt. BashArena: A Control Setting for Highly Privileged AI Agents. arXiv preprint arXiv:2512.15688. 2025.
↩
Jiaxin Wen, Vivek Hebbar, Caleb Larson, Aryan Bhatt, Ansh Radhakrishnan, Mrinank Sharma, Henry Sleight, et al. Adaptive Deployment of Untrusted LLMs Reduces Distributed Threats. arXiv preprint arXiv:2411.17693. 2024.
↩
Tim Tian Hua, James Baskerville, Henri Lemoine, Mia Hopman, Aryan Bhatt and Tyler Tracy. Combining Cost Constrained Runtime Monitors for AI Safety. In Proceedings of the Thirty-ninth Annual Conference on Neural Information Processing Systems. 2025.
↩
Aaron Sandoval and Cody Rushing. Factor (T, U): Factored Cognition Strengthens Monitoring of Untrusted AI. arXiv preprint arXiv:2512.02157. 2025.
↩
Chloe Loughridge, Paul Colognese, Avery Griffin, Tyler Tracy, Jon Kutasov and Joe Benton. Optimizing AI Agent Attacks With Synthetic Data. arXiv preprint arXiv:2511.02823. 2025.
↩
Alex Mallen, Charlie Griffin, Misha Wagner, Alessandro Abate and Buck Shlegeris. Subversion Strategy Eval: Can Language Models Statelessly Strategize to Subvert Control Protocols?. arXiv preprint arXiv:2412.12480. 2024.
↩
Mikhail Terekhov, Alexander Panfilov, Daniil Dzenhaliou, Caglar Gulcehre, Maksym Andriushchenko, Ameya Prabhu and Jonas Geiping. Adaptive Attacks on Trusted Monitors Subvert AI Control Protocols. arXiv preprint arXiv:2510.09462. 2025.
↩
Anca D. Dragan, Kenton C. T. Lee and Siddhartha S. Srinivasa. Legibility and Predictability of Robot Motion. In Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction, pages 301–308. 2013.
↩
Delano Oliveira, Reydne Bruno, Fernanda Madeiral and Fernando Castor. Evaluating Code Readability and Legibility: An Examination of Human-Centric Studies. In IEEE International Conference on Software Maintenance and Evolution, pages 348–359. 2020.
↩
Jan Hendrik Kirchner, Yining Chen, Harri Edwards, Jan Leike, Nat McAleese and Yuri Burda. Prover-Verifier Games Improve Legibility of LLM Outputs. arXiv preprint arXiv:2407.13692. 2024. ↩

Footnotes

We allow perfect auditing because we simulate a setting in which attacks are incriminating - once audited, it’s clear that they were done out of malicious intent. ↩
We report audit proportion in addition to safety because the difference in this metric between commenting and non-commenting protocols scales consistently with the trusted model’s capability, while the difference in safety doesn’t. ↩
See Section 6.1 for why we do not allow U to attempt to jailbreak T using comments. ↩
We refer to this phenomenon as a “halo effect” as a reference to the halo effect in psychology, in which a salient positive attribute (e.g., attractiveness) biases evaluators’ judgments of otherwise unrelated traits. ↩
Note that this is different than the 36.1%number we report in section 3. This is due to variance between independent runs of the model on APPS problems. ↩

Abstract PDF

Some elements are approximated — see the original for exact rendering.

source: no .bbl in archive; bibliography synthesized from .bib
macro-conditional: \thefootnote uses TeX conditionals (×3)
counter-manipulation: \setcounter stripped — displayed numbers may diverge from the PDF (×7)
counter-manipulation: \refstepcounter stripped — displayed numbers may diverge from the PDF (×3)
citations: author-year citations approximated as numeric
footnotes: \footnotetext not supported; removed (×3)

PreviousHow to prevent collusion in untrusted monitoring (guided)Next Low-stakes control (ControlConf talk)

Abstract

2 Introduction

Footnotes

We allow perfect auditing because we simulate a setting in which attacks are incriminating - once audited, it’s clear that they were done out of malicious intent. ↩
We report audit proportion in addition to safety because the difference in this metric between commenting and non-commenting protocols scales consistently with the trusted model’s capability, while the difference in safety doesn’t. ↩
See Section 6.1 for why we do not allow U to attempt to jailbreak T using comments. ↩
We refer to this phenomenon as a “halo effect” as a reference to the halo effect in psychology, in which a salient positive attribute (e.g., attractiveness) biases evaluators’ judgments of otherwise unrelated traits. ↩
Note that this is different than the 36.1%number we report in section 3. This is due to variance between independent runs of the model on APPS problems. ↩