In this module, you will learn about four main areas of verification mechanisms: hardware, cloud, intelligence, and human. Each layer has its own strengths and weaknesses, evaluated on important metrics like technical feasibility and political feasibility.
By the end of this module, you will be able to:
- Explain the relative strengths, weaknesses, current state of implementation, and most realistic path forward for hardware, cloud, intelligence, and human mechanisms, including overlaps and dependencies.
- Evaluate any verification mechanism by the claims they test, the evidence they produce, cost of implementation, deployment maturity, and principal failure modes, including actors likely to break it and why.
- Explain the confidentiality–verifiability tension and identify the most promising privacy-preserving verification mechanisms.
- Distinguish costly from cheap signals: robust mechanisms that would force an evader to attack multiple independent streams vs. weak mechanisms whose results need to be corroborated by independent sources.
Feasibility Intuitions
Before we dive in, let’s first identify some baseline intuitions: fill out the following graph with your pre-course understanding of the relative efficacy of each mechanism across four metrics:
1. Technical feasibility. Does the technical infrastructure and requisite research exist to build and run this mechanism at operable scale today? Includes technological maturity, dependencies, cost, and small-enough error rates.
Low
Fully homomorphic encryption over entire training runs, which is orders of magnitude too slow at frontier scale.
High
Compute reporting through cloud providers, as the metering and billing infrastructure already exists.
2. Political feasibility. Would the parties whose cooperation is required actually adopt and enforce it? Includes geopolitical context, incentives, intrusiveness, and confidentiality cost.
Low
International inspectors with direct access to US and Chinese frontier labs’ model weights.
High
Reporting requirements attached to existing chip export licenses.
3. Verification effectiveness. How precise, certain, and thorough is the evidence that the mechanism actually verifies? Which actors/activities does it cover? Could it distinguish training vs. inference?
Low
Voluntary lab commitments, as self-reported evidence can hide much and proves little.
High
On-chip cryptographic attestation, which proves the specific claim about the specific workload.
4. Durability. How fast does the mechanism’s viability decay—from technical progress, adversary adaptation, or political change?
Low
FLOP-threshold reporting; algorithmic efficiency gains push dangerous capabilities below any fixed threshold within a few years.
High
Mechanisms rooted in chip hardware, which takes long to mature and go obsolete.
As you’re going through this exercise, jot down in your notebook: what were the heuristics you used to evaluate whether a verification mechanism was:
- Technically feasible? → Are there historical verification precedents that have used a similar mechanism? Current technical analogs?
- Politically feasible? → How difficult would it be to get the U.S. to agree? China? What must remain confidential, no matter what?
- Effective? → What’s the minimum threshold of confidence or level of evidence a nation-state should have to ensure that a rival is compliant? What must a verifier be able to learn?
- Durable? → What develops faster, hardware or software?
Swiss Cheese: Layer Imperfect Checks
It’s important to take advantage of the unique strengths of each of the layers, while taking into account how intersection affects them as well as their specific failure modes. For example, whistleblowers and human signals may be able to give us suspicions on violations, telling us something about the scale and location of those violations. However, these signals are often complementary to the other layers, serving as confirmation or signals on what to investigate rather than independent sources of truth themselves.
The same is true of every layer. Satellite or power evidence may indicate that a large facility exists without proving what code ran there. Hardware or cloud records may describe activity precisely but still depend on trustworthy devices, signing keys, administrators, and definitions. Inspections can access evidence that remote sensing cannot, but only where inspectors have authority, access, time, and a target worth inspecting.
The Swiss-cheese model asks us to combine defenses whose holes do not line up. The goal is to combine layers that rely on different information, different actors, and different access assumptions, so that the failure of one does not automatically defeat the rest.
Evidence Taxonomies
It’s important to note that the way we’ve taxonomized verification mechanisms in this course is not necessarily the only or most accepted way to do so; on the other hand, there are several different ways you can taxonomize the evidence streams of verification, according to your writing goals and audience context. We’ve chosen the hardware/cloud/intel/human four buckets—the by-layer taxonomy—for pedagogical simplicity. Keep in mind that when going forwards, you should proactively think about the best taxonomy or categorization level for your audience; think back to the upstream and downstream exercises you completed in Module 1. For instance, you’d want to prioritize mechanisms by policy goal when speaking to congressional officials, while you’d want to focus more on the easy-to-visualize by-layer organization for educational purposes.