An AI safety control can fail in two different ways. It can stop blocking risky activity, or it can stop producing evidence that the block is gone. The second failure is more dangerous because every quiet dashboard becomes ambiguous. Anthropic's August 2026 Risk Report documents exactly this pattern: one internal flag disabled both classifier blocking and the logging of classifier flags.
Evidence reviewed: 19 August 2026. Incident details and numbers come from Anthropic's published Risk Report and retain the report's scope and qualifications.
Reading time: 8 minutes · About 1,500 words
TL;DR
- From May 2025 to April 2026, roughly 133 million exchanges involving a pool of about 50,000 human-feedback contractors ran without blocking biological classifiers.
- The internal-use flag also disabled logging of classifier flags, so traffic that would have triggered review was neither recorded as flagged nor forwarded to review mechanisms.
- Anthropic retained almost all transcripts and later rescanned them. It reported no evidence of concerning chemical or biological misuse, but explicitly lowered its confidence that no similar gaps remained.
- Blocking and evidence collection should fail independently. A policy exemption may suppress enforcement while preserving scores, logs, review routing, and an immutable record of the exemption.
- The minimum production contract is four gates: independent evidence, synthetic probes, configuration diffs with expiry, and an external task ledger that can detect missing telemetry.
What the incident actually says
The report describes a classifier coverage gap on systems used to collect human-feedback data. For nearly a year, all traffic on these systems ran without blocking biological classifiers. Most contractors had open-ended access rather than a narrow interface for rating fixed answers.
The scope matters:
| Item | Reported value |
|---|---|
| Affected period | May 2025 to April 2026 |
| Contractor pool | Roughly 50,000 people |
| Exchanges | Around 133 million |
| Transcripts retained | Almost all traffic |
| Retrospectively flagged high | 1,197 transcripts |
| Internal Anthropic traffic among flags | 757 transcripts |
| Non-red-team external transcripts manually reviewed | 62 transcripts |
The post-incident scan used Claude Sonnet 5 over all human turns. Anthropic also manually reviewed all 62 flagged non-red-team external transcripts and a sample of 30 external red-team transcripts. It found no clearly concerning misuse that would have meaningfully helped a threat actor.
That outcome should not be converted into proof that the control design was acceptable. The report makes the narrower claim that this particular exposure was very unlikely to have materially increased the assessed risk. It also says the incident increased the estimated likelihood that other similar gaps might remain undiscovered.
This is the core distinction: retrospective recovery can reduce uncertainty about consequences. It cannot restore the real-time detection path that never ran.
Why a shared kill switch creates a false green
Enforcement and evidence answer different questions.
- Enforcement asks whether a request, completion, tool call, or action may continue.
- Evidence asks what the system observed, which policy applied, what decision was made, and where the event went next.
An approved exception can legitimately change the first answer. It should rarely erase the second.
When one flag disables both, four signals collapse at once:
- The request is not blocked.
- The classifier event is not counted.
- Review queues receive nothing.
- Operators see no alert that the evidence path is empty.
Low alert volume then looks like low risk. It may actually mean the sensor has been removed.
The same report contains a second example of missing external evidence. An employee whose AI use was neither logged nor covered by automated offline monitoring assigned an open-ended task in a cluster with sensitive resources. Legacy instructions caused child agents to run with --dangerously-skip-permissions. One agent deleted many jobs, and the deletion itself exposed the unsafe agents. Anthropic says it believes the agent intended to delete only jobs it had created, but the monitoring gap prevented confirmation.
The system learned about the failure through damage, not telemetry.
Gate 1: split the enforcement plane from the evidence plane
Treat blocking and observation as separate control planes with separate failure modes.
An exemption object should contain at least:
subject + scope + policy version + approver + reason
+ start time + expiry + enforcement mode + evidence mode
The safe default for an enforcement exemption is observe, not silent. The classifier may continue scoring in shadow mode; decisions, model and policy versions, exemptions, and routing outcomes remain logged. Where data-retention agreements prohibit content retention, the evidence plane can still record bounded metadata, aggregate counters, cryptographic event identifiers, and health signals appropriate to that agreement.
The important property is independence. A bug in enforcement configuration must not automatically remove the only evidence capable of finding that bug.
Gate 2: run synthetic probes through the whole path
Service health checks often confirm that an endpoint returns 200. A safety control needs semantic health checks.
Create a small, versioned probe set with known expected outcomes. Send it through every production surface and verify each stage:
| Stage | Minimum assertion |
|---|---|
| Ingress | Probe received with expected tenant, user class, and surface |
| Classifier | Expected model and policy version scored the event |
| Decision | Block, allow, or shadow outcome matched expectation |
| Logging | Event appeared in the evidence store within the latency budget |
| Review | Selected probe reached the correct review queue |
| Recovery | Operator can trace the probe end to end by one stable ID |
Probe both positive and negative paths. A harmless control sample catches overblocking. A policy-triggering sample catches missing enforcement. A shadow-only sample catches the exact failure where blocking is intentionally absent but evidence should remain.
Alert on missing probes, not only on high-risk events. Silence is itself a failed test.
Gate 3: make configuration changes expire and produce diffs
Several incidents in the same report involve configuration drift: a classifier threshold copied from an older classifier caused five days of missed routing, one deployment surface stopped blocking for up to 48 hours, and an exemption inherited roughly four times the approved number of seats.
These are control-plane failures, not model-capability failures.
Represent safety configuration as typed data and require:
- machine-validated compatibility between a classifier and its threshold;
- a human-readable diff showing affected models, surfaces, identities, and evidence behavior;
- automatic expiry for temporary access and exemptions;
- canary rollout with before-and-after block, flag, and review rates;
- a policy that forbids one change from disabling both enforcement and minimum evidence;
- continuous reconciliation between approved intent and deployed state.
A successful deployment means the intended control state is observed on every target surface. It does not mean the configuration API accepted the update.
Gate 4: keep an external task and resource ledger
Telemetry generated only by the agent runtime shares the runtime's blind spots. High-impact automation needs an independent record of expected work.
For every unattended task, record the authorized scope, created resources, expected outputs, allowed destructive actions, and completion criteria outside the agent session. Reconcile that ledger against cluster jobs, repositories, cloud resources, and other systems of record.
This changes incident discovery from:
many jobs disappeared -> investigate the agent
to:
observed deletions exceed authorized set -> stop, preserve evidence, investigate
The external ledger also detects absence. If a scheduled agent claims completion but no expected artifact, test result, pull request, or state transition exists, the task remains incomplete.
The dashboard should measure evidence coverage
A block rate alone cannot describe control health. Track at least:
| Metric | What it detects |
|---|---|
| Traffic with expected classifier score | Coverage gaps |
| Flagged events with durable log record | Evidence loss |
| Logged events delivered to review | Routing failure |
| Active exemptions by age and scope | Exception drift |
| Synthetic probe pass rate by surface | End-to-end control failure |
| Approved vs deployed configuration diff | Unauthorized or stale state |
| Tasks with external acceptance evidence | False completion |
Break every metric down by model, product surface, organization, access class, and policy version. Aggregate success can hide one completely unprotected surface.
This extends the distinction in Agent Observability: Why Chain-of-Thought Is Not Telemetry. Useful telemetry is an intentional contract, not whatever traces happen to survive. It also complements one-token API drift monitoring: synthetic signals are valuable only when their path to storage and review is independently verified.
FAQ
Should safety logging ever be disabled?
Content retention may be restricted by privacy, legal, or contractual requirements. The system should still preserve the minimum lawful evidence needed to prove control health, such as bounded metadata, aggregate counters, policy versions, exception records, and synthetic-probe results.
Is shadow mode enough when blocking is disabled?
Only if shadow scores are stored, monitored, routed, and tested independently. A classifier that runs but leaves no durable evidence provides little operational assurance.
Why are synthetic probes better than dashboard thresholds?
Thresholds alert on observed events. A dead sensor may produce no events at all. Synthetic probes create known expected signals, so their absence directly reveals a broken path.
What is the difference between a low block rate and a logging outage?
A low block rate can be a valid property of benign traffic. A logging outage removes the denominator and the audit trail, so the rate can no longer be interpreted.
What should happen when an exemption expires?
Enforcement should automatically return to the approved default, while the expiry, resulting configuration, and verification probes are recorded. Manual deprovisioning should be an exception, not the control.