Administrator
Published on 2026-07-31 / 6 Visits
0
0

Document-Borne AI Worms: A Copilot Context Quarantine Playbook

A document-borne AI worm proof of concept shows how hidden instructions can move from an external Word file into Copilot-generated documents and then influence later workflows. The practical defense is a context-quarantine pipeline: label every source, isolate untrusted content, constrain model permissions, scan every derivative, and preserve provenance across the document chain.

Reading time: 10 minutes

TL;DR

  • A July 2026 researcher disclosure demonstrated a conditional propagation chain in Copilot for Word. The chain required a malicious document to enter Copilot context, Copilot to copy the hidden instruction into a new file, and that carrier file to be reused in another Copilot workflow.
  • The public report is a coordinated-disclosure PoC from one researcher. Microsoft has published extensive guidance on the broader indirect-prompt-injection class, while a product-specific Microsoft advisory confirming every detail of this Word scenario is not public.
  • Conventional malware scanning addresses executable payloads. This attack uses natural-language instructions that survive content extraction and influence the model.
  • Generated documents inherit the trust level of their least-trusted source until validation establishes a stronger claim.
  • The strongest practical architecture combines data marking, quarantined inference, least privilege, deterministic release gates, document lineage, and incident-response telemetry.

What the Word proof of concept actually demonstrated

Security researcher Håkon Måløy published Context Collapse, Part 3 on July 28, 2026, after a reported 144-day coordination period with the Microsoft Security Response Center and Microsoft product teams.

His disclosed scenario used a Word document containing instructions hidden as small white text. When the document entered Copilot for Word's context, the instructions affected the drafting task. In the demonstration, Copilot changed financial figures and appended the hidden instruction to the new document. A later Copilot session used that derived document without the original malicious file. The instruction triggered again and was copied forward.

That result supports a precise claim: under the reported test conditions, an attacker-controlled instruction could propagate through AI-assisted Word derivatives.

It also has important boundaries:

  • The document did not spread by opening itself or executing code.
  • A user or retrieval process had to place the source or a derived carrier into Copilot context.
  • Copilot had to follow the instruction and preserve it in the generated output.
  • A downstream workflow had to reuse the carrier document.
  • The public post does not include the complete payload, so independent reproduction from the article alone is unavailable.
  • Microsoft has not published a CVE or product advisory that independently confirms every detail of this exact propagation chain.

The term worm describes self-replication across derived documents once the required workflow conditions are present. It should not be confused with an autonomous network worm.

The mechanism has an independent research precedent. The Morris II paper demonstrated self-replicating natural-language prompts in a laboratory RAG email-assistant ecosystem. The model copied attack instructions and a malicious task from retrieved email into generated output, allowing later workflows to retrieve the new carrier. Morris II used a research-built environment rather than a commercial productivity suite. It supports the replication mechanism, while Måløy's disclosure supplies the direct Copilot for Word PoC.

Why ordinary document trust breaks

Traditional file security asks whether a document contains macros, exploits, malicious links, or executable content. An LLM workflow adds another execution surface: text that appears to be data can act like an instruction.

The disclosed hiding method illustrates the gap. White-on-white text is visually absent for a reader, yet the researcher reports that formatting is stripped before the text reaches the model. The model receives content, not the user's visual experience of the page.

Microsoft's own indirect prompt injection explanation describes the broader mechanism. Externally controlled text from webpages, email, shared documents, tool results, and other sources can be misinterpreted as legitimate instructions. Microsoft also lists hidden text, attachments, embedded content, images, PDFs, and metadata as possible carriers in its Defender for Office 365 documentation.

This changes the useful trust unit. A file can be:

  • safe to open in Word;
  • safe for a person to read;
  • unsafe to place in an instruction-following model's context;
  • unsafe to use as source material for a document that will later be trusted internally.

The fourth state creates the propagation risk. Once a legitimate employee and Copilot produce a new file, familiar ownership and location signals can make the derivative appear more trustworthy than its source.

The propagation chain has five breakpoints

Treat the attack as an information-flow chain rather than one model failure.

Stage Required condition Defensive breakpoint
1. Entry An attacker-controlled document reaches the organization External-source labeling, channel scanning, attachment policy
2. Context admission Copilot receives or retrieves the document Context allowlist, trust metadata, data marking, quarantine
3. Model influence Hidden content changes the model's behavior Prompt shields, spotlighting, plan-drift detection, task constraints
4. Derivative creation The generated file contains manipulated content or copied instructions Output diff, hidden-content scan, numeric and citation validation
5. Reuse A derived carrier enters a later workflow Provenance enforcement, release state, downstream admission policy

Each breakpoint can fail independently. A secure design therefore assumes probabilistic detectors will miss some attacks and uses deterministic controls to limit what a miss can change.

Microsoft makes the same distinction. Its MSRC guidance describes system prompts, Spotlighting, and Prompt Shields as defenses that reduce attack probability. It describes permissions, consent, data governance, and blocking dangerous actions as impact controls. The Secure Future Initiative pattern adds information-flow control, quarantined inference, least privilege, short-lived privileges, critic agents, tool-chain analysis, and human approval.

Build a context-quarantine pipeline

1. Assign trust before inference

Trust metadata should enter the workflow with the file:

document:
  id: sha256:...
  source_type: external_partner
  source_channel: sharepoint_guest
  received_at: 2026-07-31T09:15:00Z
  trust_state: untrusted
  visible_text_scan: complete
  hidden_content_scan: pending
  permitted_use:
    - read_only_extraction
  prohibited_use:
    - autonomous_edit
    - external_send
    - financial_posting

Location alone cannot upgrade trust. A document copied from an external SharePoint site into an internal folder retains its external origin. A Copilot-generated derivative retains links to every source document.

2. Compare rendered and extracted content

Run deterministic document analysis before model ingestion:

  • extract text, comments, tracked changes, headers, footers, shapes, alt text, fields, and metadata;
  • flag white-on-white, zero-size, off-page, hidden, or unusually small text;
  • compare the rendered page with the model-visible extraction;
  • normalize obfuscated Unicode and encoded blocks for inspection;
  • preserve the original hash and extracted representation.

This scanner can detect concealment patterns. It cannot prove that every remaining sentence is benign. A natural-language instruction can look like ordinary prose.

3. Separate interpretation from action

Process untrusted documents in a read-only inference zone. The model can extract facts or draft a candidate, while direct write access to shared repositories, email, financial systems, and other enterprise tools remains unavailable.

Apply data marking or Spotlighting so external content has a distinct representation. Keep the system task, user intent, and untrusted material in separate fields. Add explicit stopping conditions. If the model detects instructions inside source material, it should return a structured warning rather than decide whether those instructions are authoritative.

High-risk workflows benefit from two stages:

untrusted sources
  -> isolated extraction and classification
  -> candidate facts with source spans
  -> controlled drafting
  -> deterministic and human validation
  -> released document

The second model call receives approved facts or cited source spans, rather than the complete untrusted document, whenever the task permits.

4. Treat every derivative as untrusted

A generated file begins in quarantined_generated, not trusted_internal.

Release checks should include:

  • hidden-text and metadata scanning;
  • a visible diff of model-performed edits;
  • source-linked verification for names, dates, financial figures, and quoted claims;
  • comparison against authoritative internal records;
  • detection of new instructions, links, embedded objects, or unexplained style changes;
  • approval by a named owner for financial, legal, external, or high-impact documents.

This is where context quarantine differs from an input filter. The output can become a new input, so validation must follow the derivative.

5. Preserve lineage across reuse

Every document used by an AI workflow should produce a lineage event:

{
  "run_id": "copilot-run-...",
  "output_document": "sha256:...",
  "source_documents": ["sha256:...", "sha256:..."],
  "model_action": "draft",
  "trust_state": "quarantined_generated",
  "review": {"status": "approved", "reviewer": "role:finance-controller"},
  "downstream_locations": ["sharepoint://finance/q1/"]
}

Lineage supports admission control. A later workflow can reject an unreviewed derivative or process it in quarantine. It also makes incident response possible when a carrier has already moved through shared folders.

Contain actions even when detection fails

Microsoft's guidance explicitly recommends designing with the expectation that some indirect prompt injections will succeed. That shifts security attention from perfect classification to bounded consequences.

Use hard controls for actions with material impact:

Action Minimum control
Read an external document Read-only context, source label, extraction log
Draft a private working file Quarantined output, hidden-content scan
Modify an existing authoritative file Explicit diff and owner approval
Publish or share externally DLP, provenance check, named approval
Send email or message Recipient preview and user confirmation
Change financial or operational data Deterministic reconciliation and separate authorization
Call tools using user credentials Least privilege, short-lived grant, action allowlist

This follows the same principle as a pre-install trust boundary for coding agents: untrusted material receives minimal capability until evidence justifies promotion.

Prepare an incident-response path

If a suspicious derived document is found:

  1. Freeze the file and calculate its hash.
  2. Extract visible, hidden, and metadata content without opening it in an AI workflow.
  3. Query lineage records for source documents, model runs, edits, reviewers, shares, and descendants.
  4. Quarantine every unreviewed descendant and disable automated reuse.
  5. Search for stable indicators such as unusual hidden formatting, repeated instruction fragments, unexplained numeric transforms, and identical embedded objects.
  6. Review prompts, tool calls, document-access logs, and approval events.
  7. Restore authoritative data from an independent source.
  8. update detectors, admission policy, and regression tests with the observed pattern.

Microsoft's prompt-abuse detection playbook recommends inventory, prompt and data monitoring, conditional access, DLP, audit logs, investigation, and continuous oversight. Document lineage supplies the missing relationship between those events.

Measure the control system

Useful metrics follow the propagation chain:

  • percentage of external documents carrying source and trust metadata;
  • percentage scanned through both rendered and extracted views;
  • percentage of AI-generated documents with complete source lineage;
  • high-impact edits with deterministic reconciliation;
  • quarantined derivatives released without named approval;
  • downstream workflows that rejected an unreviewed source;
  • time required to enumerate all descendants of a suspicious document;
  • prompt-injection detections by channel, source, model, and action;
  • false-positive rate and review latency for each control.

Track prevention and containment separately. A detector can miss a prompt while least privilege still blocks an external send. Combining both into one pass rate hides which control actually protected the organization.

Frequently asked questions

Is this a conventional Microsoft Word virus?

No executable malware was required in the reported demonstration. The payload was natural-language content interpreted by Copilot, and propagation depended on AI-assisted creation and later reuse of carrier documents.

How does Microsoft protect Copilot from prompt injection?

Microsoft documents layered controls including hardened prompts, Spotlighting, Prompt Shields, grounding boundaries, output filtering, Defender channel inspection, information-flow control, least privilege, audit, and user approval. Microsoft also describes indirect-prompt-injection detection as an open research challenge, so the architecture relies on multiple layers.

Can Defender for Office 365 stop the Word propagation chain?

Defender's documented prompt-injection protection inspects inbound email before delivery and can reduce one entry path. Documents already present in OneDrive, SharePoint, Teams, partner repositories, or other channels still require context admission, output validation, lineage, and action controls.

Why can Copilot read text a person cannot see?

Models often receive extracted document content rather than a pixel-perfect representation of the page. The disclosed PoC used visually hidden text that remained present in the extracted content.

What should organizations do immediately?

Label external documents, restrict their use to read-only Copilot workflows, scan rendered and extracted content, quarantine generated derivatives, require approval for high-impact edits and sharing, and begin recording source-to-output lineage.

References


Comment