AI model distillation defense cannot be evaluated by the number of banned accounts. Anthropic reports that new controls reduced exposed reasoning tokens by 90% to 99%, while also changing integrations, occasionally blocking benign requests, and charging for some blocked traffic. A defensible program needs three ledgers in the same experiment: capability leakage, legitimate-user friction, and false-positive harm.
Evidence reviewed: October 13, 2026. Anthropic's reduction and false-positive figures are internal measurements and have not been independently reproduced. Anthropic also states that it has not yet observed a highly sophisticated, persistent attempt against the new defenses.
Distillation attacks target an information advantage
Knowledge distillation is a legitimate model-development technique when a developer trains a smaller student model on outputs from an authorized teacher. A distillation attack uses model access without authorization to reproduce protected capabilities, often through coordinated traffic and many accounts.
The security objective is therefore broader than stopping suspicious prompts. The defender must reduce the useful information that an attacker can acquire per unit of money, time, identity, and infrastructure.
Anthropic's new defense report focuses on hidden or protected reasoning because it can teach a student more than final answers alone. In one internal experiment, a pretrained Claude model trained on full Claude Opus 4.8 transcripts scored 80.4% on SWE-bench Verified. Training on summarized reasoning produced 71.8%. This is author-reported evidence for that experiment, not a universal estimate of the value of reasoning traces.
Attackers adapted. Anthropic describes extraction attempts spread across proxy networks and many accounts. OpenAI reported a separate coordinated campaign in which activity peaked at 16,000 relevant-pattern requests from more than 4,000 users over two days; its wider investigation found a related cluster of more than 15,000 users. OpenAI explicitly notes that those figures describe attempts, not necessarily successful extraction.
The operational lesson is clear. A single-request classifier is one sensor inside a campaign-level problem.
Anthropic's layered defense
Anthropic describes three control families.
First, it restricts where reasoning artifacts can travel. Preserved thinking prevents newer accounts from editing prior context on selected models. Reasoning is also constrained across model generations and, for Claude Sonnet 5.5, to the organization that generated it.
Second, a two-stage classifier identifies reasoning-extraction attempts. Anthropic says it used internal and external red teaming and reports a false-positive rate below 0.1% on the protected areas.
Third, repeated attempts trigger stricter limits and account enforcement. Anthropic also resumed charging for requests blocked in areas where it reports a false-positive rate below 0.1%, increasing the attacker's cost of repeated exploration.
After the controls were deployed, Anthropic estimates that extracted reasoning tokens on its own platform fell by a factor of 10 to more than 100. It reports banning tens of thousands of associated accounts, often within two hours of campaign start.
These results matter, but they answer different questions. The token reduction estimates leakage. The false-positive figure estimates one category of user harm. Time to ban estimates response speed. Account count measures enforcement volume. Combining them into a single success rate would hide the trade-offs.
The three-ledger evaluation contract
Ledger 1: Capability leakage
Count information value, not only blocked requests. A useful test set should cover ordinary final answers, summarized reasoning, protected artifacts, conversation transfer, model transfer, tool outputs, streaming paths, and partner-hosted deployments.
Measure at least:
- exposed protected tokens per million requests;
- successful extraction rate by technique family;
- capability uplift of a controlled student trained on captured data;
- time and cost required to reach a fixed student performance;
- leakage through first-party, cloud-partner, and reseller paths;
- recurrence after a defense update.
The student-model test is important. Two datasets can contain the same number of tokens and carry very different training value. Conversely, a large token reduction may not produce a proportional reduction in capability transfer if the remaining tokens are unusually informative.
Ledger 2: Legitimate-user friction
Security controls change the product surface. Anthropic acknowledges that preserved thinking required some developers to update integrations and that blocked requests can now incur charges. These effects belong in the evaluation, not in a footnote.
Track:
- migration failures and support contacts;
- added latency and token cost;
- interrupted cross-account and cross-organization workflows;
- percentage of blocked requests that require appeal;
- time to restore legitimate access;
- revenue rejected or delayed;
- developer behavior that shifts to less secure channels.
Friction can also weaken security. If a compliant workflow becomes difficult, users may export artifacts, share credentials, or route traffic through intermediaries. The defense should therefore be evaluated across the full workflow, not only at the model endpoint.
Ledger 3: False-positive harm
A rate below 0.1% can still produce many incidents at frontier-platform scale. Report both the rate and the absolute count, with confidence intervals and segmented results.
Important slices include language, country, account age, organization size, domain, tool use, long conversations, accessibility workflows, and legitimate security research. Measure severity too. A delayed response, a charged block, an account freeze, and organization-wide suspension have different costs.
The acceptance test should include recovery:
false-positive harm
= detection error rate
× enforcement severity
× time to correction
× affected workflow value
This prevents a low classifier error rate from masking a slow or punitive appeals process.
A defense architecture that can be verified
Bind artifacts to context and identity
Protected reasoning should carry provenance that the service can validate across conversations, accounts, organizations, models, and cloud partners. The policy should state which transfers are valid and which require re-authorization.
Combine content and behavior signals
Content classifiers see individual attempts. Behavioral systems see repetition, account coordination, payment reuse, network overlap, capability concentration, and changes after blocks. Neither layer is sufficient alone.
Escalate enforcement with evidence
Use a graduated response: observation, challenge, lower limits, artifact restrictions, temporary hold, investigation, and account or network enforcement. Stronger actions require stronger evidence and a clearer recovery path.
Share indicators without sharing sensitive content
OpenAI says partner-hosted services and tool-output paths need equivalent protection. Providers should exchange campaign indicators, artifact formats, and control lessons through trusted channels while minimizing exposure of customer content and defensive internals.
Keep a frozen regression suite
Every update should run against four fixed groups:
- known extraction techniques;
- adaptive variants unseen during training;
- high-value legitimate workflows;
- ambiguous requests requiring escalation.
Publish changes in leakage, friction, and false positives separately. A defense that improves one ledger by silently damaging another has not passed.
This is also how to interpret the safeguard tax discussed in our Claude Fable and Mythos benchmark audit. The deployable system includes classifiers, artifact policy, fallbacks, access programs, and recovery procedures. Model weights alone do not determine user-visible capability or security.
Release gates for distillation defenses
A production release should require:
- A statistically bounded leakage reduction on a frozen test set.
- A student-uplift experiment showing that the remaining data is materially less useful.
- False-positive rates and absolute counts across critical user segments.
- Latency, cost, migration, and support-impact budgets.
- A tested appeal and restoration objective for each enforcement tier.
- Equivalent controls across first-party and partner-hosted inference.
- A rollback path if a defense harms legitimate traffic.
- Ongoing canaries because attackers adapt after deployment.
The final metric is not accounts banned. It is useful capability transferred per attacker resource, constrained by an explicit budget for legitimate-user harm.
Frequently asked questions
What is an AI distillation attack?
It is the unauthorized use of a model's outputs or protected reasoning to train, reproduce, or improve another model, often at coordinated scale and in violation of access terms.
Is defensive distillation the same thing?
No. Defensive distillation is an older robustness technique that trains a model using softened outputs from another model. The phrase distillation defense here means controls against unauthorized capability extraction.
Why is hidden reasoning more valuable than final answers?
Reasoning traces can expose intermediate strategies and decompositions that provide richer training supervision. The magnitude of that advantage depends on the model, task, trace quality, and student-training setup.
Can a low false-positive rate guarantee minimal user impact?
No. Impact also depends on request volume, which users are affected, enforcement severity, charges, and recovery time. Report absolute incidents and workflow harm alongside the rate.