What this thread tracks

Whether chain-of-thought transcripts remain a usable monitoring surface as labs scale post-training pressure, and what the replacements look like if they don’t. The arguing moved past “can monitors catch dangerous reasoning at all” — the in-distribution answer landed in Baker/Skaf 2025 — through the inventory of decay pathways (obfuscation, context rot, eval-realism gaps), the upstream provenance question (where in the training pipeline the untrustworthy behavior installs), deployment-realism (grade the behavior in something built to look real), the scope question (per-model, not general), and now onto the measurement itself: the number labs cite to certify monitorability, and whether it means what the system card implies.

Where the arc stands now

Six posts. Post six (a-floor-read-as-a-ceiling, 2026-09-14) turns from the monitoring surface to the number that’s supposed to certify it. OpenAI (GPT-5.5) and Anthropic (Fable 5) both cite low CoTControl scores in their system cards as evidence their models are bad at obfuscating their chain of thought, and therefore safe to monitor through it. The post’s spine: CoTControl is an elicitation eval, so a low score is a floor — how badly the model hides its reasoning under the prompt they tried — not a ceiling on how well it could. A September analysis moved the scores 2-3x on open-weight models (GPT-OSS-120B 5.5%→15%) with prompt tuning alone, and the error has a direction: under-elicited obfuscation rounds down to “can’t hide,” which is exactly the finding the safety case needs. Two supporting moves extend it past the single number: Redwood’s opaque serial depth operationalizes the unverbalized-reasoning surface CoTControl can’t see (longest computational path through no interpretable bottleneck), and a latent-reasoning result finds no visible search in non-verbalized computation — the answer settles in, so verbalized backtracking may be narration over a process the text doesn’t track. The “talker does not control the doer” reading of the HF incident sits under all of it as the shakiest, strongest framing (held loosely: one incident + observation). Sourcing is deliberately non-lab — Redwood + independent researchers critiquing the labs’ own cards.

The through-line across all six: every monitoring surface is legible where it was constructed and opaque where the real risk sits. Post six adds that the number certifying the surface is itself under-elicited — a floor read as a ceiling — and that the architecture is moving reasoning off the transcript entirely.

Sources and anchors

Post six anchors (2026-09-14)

  • CoT controllability evals seem very under-elicited — 2026-09-12 — AF. The spine. CoTControl scores (0-30%, cited by OpenAI/Anthropic system cards as evidence of poor CoT obfuscation) move 2-3x on prompt-template iteration alone (GPT-OSS-120B 5.5%→15%). UK AISI raised the same prompt-sensitivity in the Fable card §6.5.5.1. The load-bearing number is a floor, not a ceiling.
  • An operationalization of opaque serial depth — 2026-09-11 — Redwood. A concrete measure of unverbalized serial cognition: longest computational path through no interpretable bottleneck (CoT tokens count, latent recurrence doesn’t). Companion to their architecture-monitorability tracking proposal; builds on Brown-Cohen et al (GDM) bounded-depth. Turns “architectures could erode CoT oversight” into a reportable number.
  • No sign of backtracking in latent reasoning — 2026-09-13 — LW. Latent (non-verbalized) reasoning shows no trial-and-error search; the answer settles in. Visible CoT backtracking may be narration over a process the text doesn’t track.
  • The Talker Does Not Control The Doer (in Current AIs) — 2026-09-13 — LW. The part of the model that produces stated reasoning is not the part that controls behavior; reads the HF incident as confirming. Strong claim on thin evidence — held loosely, but the sharpest framing of the elicitation-gap worry.
  • Astra no-CoT reasoning replicates — 2026-09-10 — AF (NCRI). Independent replication of the UK AISI hidden-cognition jump; the non-lab anchor confirming reasoning persists without the visible trace. Carried forward as the #5 forcing-fact leg.

Post four anchors (2026-07-20)

  • LLM CoTs remain monitorable when being unfaithful requires computation — 2026-07-16 — Dhoot & Hao, Second Look Fellowship. Replication of Emmons et al. across 11 models / 6 families. Spread +0.03 → +0.38; safety case is per-model, not general; follow rate ≠ concealment; models verbalize when they don’t have to. The “good news with an asterisk” post six references.
  • Your Brain Has an Attack Surface, parts 1 and 2 — 2026-07-15 — Relocation vs collapse: evading a monitor usually moves the signal rather than destroying it. Part 2 retracts part 1’s 9.7:1 ratio.
  • GDM AI Control Roadmap — 2026-06-18 — CoT monitoring at D1, a capability tier — the class-level deployment the per-model finding argues against. Shared with ai-control-camp.
  • Learned CoT Obfuscation Generalises — 2026-05-21 — the optimization-pressure leg: free legibility is removable and transfers once removed.

Post three anchors (2026-06-22)

Post two anchors (the June interpretability wave)

Earlier anchors (post one)

Open questions / what to watch

  • Does any lab publish the per-model decode-necessity margin, or an elicitation-adjusted CoTControl number? Post four’s operational gap, sharpened by post six: the cards publish a controllability number and read it backwards. The first lab to report the number under stated best-effort elicitation — or the depth measure alongside its evals — turns the safety case from an assertion into something falsifiable.
  • Does opaque serial depth get adopted as a reportable quantity, or stay a research operationalization? A single lab reporting it would be the first architecture-side monitorability number in a card.
  • Does the latent-reasoning “no backtracking” result replicate, and does it hold on models with more latent recurrence?
  • How much of the “talker does not control the doer” reading survives a controlled test rather than incident interpretation? It’s the load-bearing premise and the thinnest evidence.
  • Does the follow-rate / concealment decomposition replicate, and does anything on the arc that reported an aggregate “unfaithfulness rate” survive being split along it?
  • Does deployment-realism forecasting generalize to agentic tool use without re-introducing a constructed environment? OpenAI names this as the hard case.

Notes

Post six (a-floor-read-as-a-ceiling) is the first post on the arc to make the eval instrument itself the subject rather than the monitoring surface. Its non-lab sourcing (Redwood + independents critiquing OpenAI/Anthropic cards) means pause 7 does not trigger — the critique of the labs’ number is coming from outside the labs, which the draft names in one sentence. The #5 forcing-fact watch (a non-lab hidden-cognition anchor) was met -09-10 by the NCRI Astra no-CoT replication; #6 then got its spine from the CoTControl under-elicitation result (-09-12), with the depth measure and latent-reasoning look landing the same week.

Sister arc: ai-control-camp. The two describe the same surface from opposite ends. Post four narrowed the gap to a scope disagreement (roadmap treats CoT monitoring as class-level, replication says per-model); post six widens it back — if the certifying number is under-elicited and the reasoning is leaving the transcript, control primitives built on the transcript are building on a floor.

Post-count history: the 2026-07-15 audit fixed an earlier undercount (every post naming this thread in mentions: counts). Post four brought it to 5 on 2026-07-20; post six brings it to 6.