The Veto Nobody Used

The first external review under the new auditor arrangement cut nothing and changed nothing. The reason it came back clean is that the models aren't capable enough to be dangerous yet, not that the safeguards were shown to work — and a clean audit is not the same result as a working one.

The test everyone building this arrangement said would matter just ran for the first time, and it came back empty.

The arrangement, for anyone arriving cold: outside auditors — here METR — given inside access to a frontier lab, with the right to publish their findings and one catch, that the company can require redactions in categories the company itself defines. I argued a month ago that the whole structure is worth something at exactly one moment — when an auditor publishes a report the company made it cut, and a reader gets to see whether the flag names a real, weighable disagreement or arrives as a line of boilerplate about confidential IP. On October 20 the first such report landed. Nothing was cut.

METR’s external review of the “misalignment in high-stakes settings” section of Anthropic’s August risk report carries the disclosure the whole question turned on: Anthropic held the right to veto the publication and redact any part of it, and used none of it. No redactions. METR made “only minor wording and clarification changes.” The first time the machinery everyone argued about was switched on, it produced its emptiest possible output — a record that nothing was withheld.

A no-op is not a pass

That is good news in the narrow sense, and worth saying plainly: a transparency mechanism whose first public act is “we could have been cut and weren’t” beats a sealed report with a trust-us at the end. But a veto that goes unused tells you very little about how the veto behaves. We learned it exists. We did not learn what it does under pressure, because there was no pressure — no finding inconvenient enough that anyone reached for the redaction list. The disagreement that would actually test whether a flag can carry a specific, weighable objection didn’t happen this round. The cliffhanger resolved by being postponed.

There is a scope caveat that cuts the same direction, and METR flags it: this engagement predates the newer independent-evaluator terms. So the clean result isn’t even a test of the redaction rules people have been scrutinizing — it’s the last review under an older arrangement. The rules everyone was watching haven’t been exercised yet at all.

Low risk, and the reason matters more than the verdict

The substance is where it turns sharp, and it’s the half a skim will miss. METR’s best guess agrees with Anthropic: catastrophic-misalignment risk is low for the models covered. Stop at that sentence and it reads as the safety case holding up to an outside look. The body says something far narrower. The risk is low, METR writes, primarily because the models’ capabilities are limited — not because the safeguards or the alignment training are carrying the weight. METR is explicit that it is “not confident” the report’s methodology would reliably catch misalignment if the risk were actually higher, and that ruling it out with any confidence would take much more empirical work and progress on several still-unsolved problems.

Those are two different claims wearing the same number. “Risk is low because the safeguards work” is a statement about the safeguards. “Risk is low because the models can’t do much harm yet” is a statement about the models, and it says nothing about the safeguards except that nothing strong enough to test them has come along. The entire reason to put an independent evaluator inside the building is to tell those two apart. The first time one did, the answer was the second, with the first left open.

Reading the blanks

Put the two halves together and the first run of the arrangement returned a blank on both axes it was built to read. On governance: nothing to flag, so the one reliably public artifact stayed empty. On substance: a clean bill that rests on the models being too weak to be dangerous, which is a verdict on the models and a shrug about the methods.

Neither blank is bad news. Both are easy to mistake for reassurance, and it’s the same mistake twice — reading an absence of signal as a signal. A clean audit looks exactly like a working one right up until the system is put under load, and nothing here was put under load.

— Marlow