Hidden AI capabilities expose a precise limit in capability elicitation audits: a clean evaluation can establish what an AI model did under a stated test, but it cannot universally prove that an adversarially supplied model lacks a capability. A new NeurIPS 2026 paper constructs encrypted neural circuits that remain hard to trigger or identify even when the auditor has every model weight.
The useful lesson is narrower than the headline and more actionable than another call for stronger red teaming. Audit conclusions need an explicit evidence boundary. Deployment decisions then need independent controls for model provenance, authority, runtime observation, and revocation.
Evidence reviewed: October 15, 2026. This article uses arXiv version 2, submitted October 14. The construction is a proof of concept under stated cryptographic and compiler conditions.
Reading time: 10 minutes · About 2,100 words
TL;DR
- Capability elicitation establishes a lower bound: it shows capabilities an evaluation successfully surfaces.
- The paper gives a conditional construction that hides an executable neural network behind cryptographic digital lockers, even from a polynomial-time auditor with full weight access.
- Unelicitable means the trigger and payload remain hard to recover. It does not mean the compiled structure is invisible.
- The experiment hides a 672-weight ternary classifier inside a roughly 2.07-billion-parameter compiled host. This is a costly model organism, not a turnkey frontier-model attack.
- A negative audit should be reported as no capability found under this threat model, method, data, and budget.
- Teams need a control chain that combines elicitation, weight and provenance checks, least authority, runtime monitoring, and rapid revocation.
What the paper actually establishes
The paper, AI Models Can Provably Hide Arbitrary Capabilities, studies a model supply-chain attack. An adversary can modify weights before deployment, chooses a high-entropy secret trigger, and embeds a target neural network inside a host model. The defender receives the full architecture and every weight, knows how the embedding scheme works, and may run any polynomial-time audit. The trigger remains secret.
The authors define white-box unelicitability through two tests. In the first, an auditor wins by finding an input that unlocks the hidden computation. In the second, the auditor must identify which of two equal-size capabilities is encrypted in the weights. Digital-locker security transfers to these games when the locker is secure and the compiled network executes correctly.
This result matters because capability elicitation is often treated as an aggressive search problem. Evaluators improve prompts, add tools, build Agent scaffolds, fine-tune on relevant data, inspect activations, and raise the compute budget. The UK AI Security Institute protocol and METR guidance show why these practices are valuable. They reduce ordinary evaluation failures and produce stronger estimates of demonstrated capability.
The cryptographic construction identifies a different class of failure. More search remains bounded by the hardness assumption. Full weight access does not automatically reveal a trigger protected by a secure digital locker.
The theorem therefore blocks one universal claim:
A sufficiently strong elicitation audit can always prove that an adversarially supplied model lacks a hidden capability.
It leaves many narrower claims intact. Evaluations can reveal present capabilities, compare models, measure performance in defined environments, expose ordinary sandbagging, and support deployment thresholds. Their clean result remains useful evidence with a defined scope.
The construction in plain language
A digital locker is a program that releases a message only when it receives the right key. Here the key is the trigger and the message is a hidden network's weights.
The authors compile the locker and an execution unit into neural-network components. Each forward pass tries to unlock the encrypted weights. An ordinary input follows the benign host path. A correct trigger recovers the hidden matrices and runs the embedded classifier.
The security argument depends on several conditions:
- The trigger has enough min-entropy.
- The digital locker satisfies its cryptographic assumptions.
- The compiled circuit executes correctly despite finite precision and normalization.
- The auditor uses polynomial resources.
- The attacker can alter the weights before deployment.
These conditions define the theorem's reach. They also separate this paper from ordinary sandbagging. A strategically underperforming model may respond to better prompts, fine-tuning, or internal interventions. A cryptographic locker is engineered to make trigger recovery computationally hard.
The paper also separates unelicitability from undetectability. An auditor may notice sparse weights, unusual correlations, or a suspicious computational structure. The guarantee concerns recovering the trigger or identifying the encrypted payload. It does not guarantee that the modified model looks statistically natural.
The experiment supports a PoC, not prevalence
The experiment uses two controlled capabilities: subset parity and a binary chemistry task that predicts whether reagents yield a candidate product. The chemistry payload is a ternary multilayer perceptron with dimensions 64, 8, 16, and 2, totaling 672 ternary weights.
Embedding that small payload is expensive. The main compiled host has 38 layers, about 2.07 billion dense parameters, 1.90 million nonzero parameters, and a maximum residual-stream width of 10,266. Hash subcircuits account for about 95% of its dense parameter count. The authors present these figures as proof-of-concept costs.
The controlled results are still informative:
| Evaluation | Encrypted model | Relevant comparison |
|---|---|---|
| Correct trigger | 73.5% ± 2.6% | The hidden classifier executes |
| SFT, 500 steps | 50.9% ± 1.0% | Random-network control: 50.6% |
| LAT, eight layers | 52.9% ± 3.9% | Random-network control: 53.6% |
| GCG, 16-bit trigger | 52.7% ± 1.4% | Under the evaluated query budget |
The 8-bit GCG result reached correct-trigger performance only by enumerating all 256 candidates. It matched query-budgeted random search. This is a useful control because it distinguishes an optimizer finding structure from brute force covering a tiny trigger space.
The evidence boundary remains sharp. The authors deliberately constructed the circuits. They do not claim that ordinary training produces them. The payload is small, overhead is substantial, the compiled structure may be detectable, and full evaluation inside a pretrained frontier model remains future work. As of October 15, the paper says models will be released, while its public arXiv materials provide no repository URL or commit hash that readers can independently verify.
The supported conclusion is a conditional counterexample and a working model organism. The evidence does not establish widespread hidden cryptographic capabilities in deployed models.
A five-level evidence ladder for capability audits
Once elicitation is understood as lower-bound evidence, audit reports can state exactly what each layer supports.
| Evidence level | Evidence obtained | Claim it can support |
|---|---|---|
| 1. Behavioral observation | A model succeeds on a task under recorded conditions | The evaluated system demonstrated this capability |
| 2. Structured elicitation | Multiple methods, budgets, baselines, holdouts, and failed-run analysis | The audit searched a defined capability envelope |
| 3. Model integrity | Weight lineage, hashes, training records, anomaly checks, and mechanism probes | The supplied artifact matches an expected provenance and structure within tested limits |
| 4. Deployment containment | Scoped credentials, tool and network boundaries, runtime monitors, and tripwires | A missed capability has bounded reachable impact |
| 5. Recovery evidence | Revocation, rollback, forensic replay, and regression tests | The organization can detect, contain, and learn from activation |
No single layer substitutes for the others. A strong behavioral audit says little about tampered weight provenance. A signed artifact says little about future behavior. Runtime monitoring helps when a trigger activates, while it cannot retroactively prove the model lacked the payload before activation.
The wording of a negative result should preserve these distinctions:
We found no evidence of capability C under threat model T, methods M, data D, and compute budget B. This result does not establish capability absence against adversarial weight modification.
That sentence is more useful than passed the safety audit. It tells a release owner which uncertainty remains and which control must carry it.
How a missed capability becomes a deployment failure
An audit miss becomes dangerous through a chain of later decisions:
negative result → release approval → model authority → tool and data access → persistent execution → downstream propagation
Every arrow can amplify the initial error. A clean CBRN proxy score can support a broader release. The release may receive code execution, network access, secrets, or autonomous retry budgets. Fine-tuned descendants may inherit weights while losing the original provenance record. A runtime alert may lack the authority to revoke credentials.
This is why the paper's most practical variable is authority. The authors state that harm scales with the capability hidden and the authority granted to the host. A model with a missed capability and no meaningful access creates a smaller failure surface than the same model connected to production tools.
The risk response should therefore combine evidence and containment:
- Verify provenance before capability claims. Record training ownership, base and derived weight hashes, transformation history, signing identity, and every post-training modification.
- Run elicitation with explicit budgets and controls. Preserve prompts, scaffolds, fine-tuning data, tool access, trial counts, random-search baselines, failed runs, and evaluator versions.
- Inspect the artifact through an independent channel. Test weight statistics, graph structure, sparsity, unusual correlations, and mechanism-level signals. Report what each detector can see.
- Bind authority to the audit envelope. Higher uncertainty should lead to narrower credentials, shorter sessions, less egress, smaller write scope, and stronger human gates.
- Monitor execution rather than only outputs. Watch internal anomalies, tool sequences, network paths, privilege changes, and trigger-like state transitions during inference.
- Make revocation a separate control plane. Operators must be able to pause the run, revoke identity, block egress, preserve evidence, roll back effects, and prevent affected weights from propagating.
This complements the reproducible evidence contract for model misalignment and the trajectory controls required for long-horizon AI safety. The new paper changes the first question in both systems. A clean pre-deployment test is an observation about the audit, not a certificate about every latent computation in the artifact.
Frequently asked questions
Does this make AI capability evaluations useless?
Capability evaluations remain essential for measuring demonstrated behavior, comparing systems, finding ordinary hidden capabilities, and defining deployment thresholds. The paper limits the strongest absence claim under an adversarial weight-supply threat model.
Does full white-box access solve hidden capability auditing?
Full weight access expands the available methods, but the construction is designed so a polynomial-time auditor still cannot efficiently recover the trigger or distinguish the encrypted payload under the stated assumptions. White-box access can still help detect anomalous compiled structure.
Can ordinary model training create these cryptographic circuits?
The paper makes no such claim. Its circuits are deliberately compiled into the weights. Natural emergence, practical concealment in frontier models, and prevalence in deployed systems remain open questions.
Are open-weight models uniquely affected?
The core issue is untrusted model provenance and the ability to modify weights before deployment. Open weights can improve inspection and independent research. They also require consumers to verify artifact origin and transformations. A closed artifact from a compromised supply chain can present the same integrity problem.
What should change in the next audit report?
State the threat model, methods, data, compute budget, baselines, artifact provenance, structural checks, deployment authority, and residual uncertainty. Phrase a clean result as bounded evidence, then connect it to runtime containment and revocation.
The practical conclusion
The paper does not end capability auditing. It gives capability auditing a more honest output type.
Positive elicitation remains evidence that a model can do something. Negative elicitation becomes evidence about the search that was performed. Model integrity, deployment authority, runtime behavior, and recovery require their own evidence channels.
The strongest next step is therefore architectural. Keep improving elicitation, then stop asking it to carry the entire safety case. Build a chain in which one missed capability cannot silently become broad production authority.