An AI slop CVE can look ready for automation: an identifier, affected versions, technical prose, a proof of concept, and a severity score. None of those fields proves the vulnerability exists. Before an agent changes code or upgrades production dependencies, require a reproduction contract that binds the claim to real source, a reachable path, an isolated failing test, and an authentic fix.
The SQLite case was a pipeline failure
On July 30, 2026, JFrog published an audit of six SQLite CVEs. The reports had entered public vulnerability data flows with High or Critical metadata. JFrog checked the claimed SQLite versions, functions, line numbers, patches, and proof-of-concept inputs. The claims collapsed under direct inspection.
The failures were concrete:
- one advisory named a function that did not exist in the claimed SQLite version;
- another described a patch between versions even though the relevant source file had no change;
- cited line numbers in one report exceeded the length of the target file;
- several PoCs failed during parsing or returned normal output before reaching the alleged vulnerable logic;
- one report used a function signature that did not exist in the source.
JFrog checked out the named SQLite tags, built clean copies in Docker, ran the submitted inputs under AddressSanitizer, and audited NVD, CPE, and GHSA metadata. SQLite's own CVE page now lists the six records as not being SQLite bugs, describes them as unreproducible, and says they appear to be AI hallucinations.
JFrog then reviewed the same GitHub account's wider set of 55 advisories. Its classification was 54 fabricated advisories and one real bug wrapped in unverified CVE metadata. That 54-to-1 result is JFrog's research conclusion, not an official CVE Program classification.
The status also changed after JFrog published. By July 31, the official CVE API showed all 55 IDs as REJECTED. Rejection does not mean that every underlying software bug was imaginary. It means those records no longer support treating the described issues as valid CVEs.
A CVE record has several evidence layers
Security automation often treats a database row as a verified vulnerability. The ecosystem does not make that inference safe.
| Evidence layer | What it proves | What it does not prove |
|---|---|---|
| CVE ID assigned | a CNA created a record identifier | independent reproduction |
| record published | required record fields and references were published | vendor confirmation or exploitability |
| NVD or ADP enrichment | another system added CPE, CWE, CVSS, or decision metadata | the PoC reaches vulnerable code |
| maintainer corroboration | the affected project recognizes the issue | your deployed configuration is exposed |
| isolated reproduction | the claimed version fails under recorded conditions | the proposed patch is correct |
| verified fix | the failing case becomes safe without regression | protected code is deployed everywhere |
The six SQLite records demonstrate why these layers must remain separate. Severity metadata appeared before source and PoC verification. A scanner could therefore create urgent tickets or trigger a patching agent even when the named code path did not exist.
The CVE CNA rules require a vulnerability determination and public references. They do not turn every published record into an independently reproduced exploit. A working PoC is valuable evidence, but it is not a universal publication precondition.
Use a six-gate reproduction contract
Raw vulnerability feeds should enter quarantine. Promotion into auto-remediation should require explicit evidence at each gate.
Gate 1: Provenance and current state
Record the assigning CNA, publication history, references, and current CVE state. Refresh the state immediately before action. A record that changed to REJECTED, DISPUTED, or materially revised must invalidate the previous plan.
Vendor absence is a warning, not a verdict. Unknown provenance, contradictory version ranges, placeholder product data, and rapidly changing severity increase the review level.
Gate 2: Source identity
Freeze the exact upstream repository, tag or commit, build configuration, file, function, and claimed line. Verify that every named symbol existed in that version.
This gate would have stopped several SQLite reports before compilation. A function introduced years after the claimed target version cannot support the described path. A line beyond end of file is not a minor citation error when the advisory depends on that code location.
Gate 3: Reachability and threat model
A software bug becomes a security vulnerability only under relevant attacker capabilities and deployment conditions. Confirm that untrusted input can reach the alleged sink, the affected feature is enabled, and the stated impact grants a capability the attacker does not already possess.
SQLite emphasizes this boundary in its official vulnerability guidance. Many SQLite crash reports assume an attacker can already execute arbitrary SQL or supply a hostile database. That precondition rarely matches ordinary applications and changes the security impact.
Gate 4: Isolated reproduction
Build the exact version from clean source inside a restricted environment. Enable relevant sanitizers and logging. Run the PoC unchanged before attempting repairs. Preserve:
- source and build hashes;
- compiler, flags, dependencies, and platform;
- PoC hash and invocation;
- exit code, stdout, stderr, sanitizer output, and stack trace;
- repeated-run results and a negative control.
A missing or failing PoC does not always prove that the vulnerability is false. Environment assumptions may be incomplete. It does mean the item has not earned automatic code changes. Route it to human investigation with the missing evidence listed.
Gate 5: Fix authenticity and causality
Verify that the referenced fix commit or pull request exists, touches the claimed path, and appears in the stated fixed release. A version number in an advisory is insufficient.
The strongest regression has two sides: the frozen vulnerable build fails under the reproducer, while the fixed build passes the same test. Add boundary cases to detect a patch that merely filters the supplied input. Review unrelated behavior and security invariants before accepting the change.
Gate 6: Action policy and deployment proof
Automatic remediation should be limited to cases where the evidence contract is complete, the change is reversible, the patch source is trusted, and the rollout has bounded blast radius. Otherwise choose one of three outcomes: quarantine, request more evidence, or require human approval.
After deployment, verify the running version, rescan the affected asset, repeat the negative test where safe, and monitor rollback thresholds. A generated patch is a proposal. A protected deployment is the completion state.
Make the gate machine-readable
A small record is more useful than a long natural-language assurance.
vulnerability_gate:
cve: CVE-YYYY-NNNNN
state: PUBLISHED
checked_at: 2026-08-04T08:00:00Z
source:
repository: https://example.org/upstream/repo
commit: 4d3c2b1
function_present: true
exposure:
feature_enabled: true
untrusted_input_reachable: true
reproduction:
environment_sha256: a921...
poc_sha256: 88c4...
attempts: 3
sanitizer_failure: true
fix:
upstream_commit: 771e...
vulnerable_fails: true
fixed_passes: true
decision: allow_canary
The remediation executor should reject absent, stale, or contradictory fields. Bind approval to the exact package, version, target assets, and patch hash. Log every rejected candidate as eval data. That converts false advisories from recurring toil into regression tests for the security pipeline.
This gate complements a broader AI vulnerability pipeline. The earlier article tracks risk from discovery through installed protection. The reproduction contract here decides whether an external claim deserves to enter that lifecycle at all.
Avoid the AI-detector shortcut
JFrog used GPTZero as one signal that the advisory prose appeared AI-generated. That signal is not the basis for rejecting a vulnerability. Text detectors can misclassify human and machine writing, and a human-written report can still be technically false.
The correct question is not who wrote the prose. It is whether the named code exists, the path is reachable, the input reproduces the failure, the impact matches the threat model, and the fix changes the outcome. Those checks remain valid regardless of authorship.
FAQ
What is an AI slop CVE?
It is an informal label for a low-quality or fabricated vulnerability report whose technical details may have been generated or amplified by an LLM. The label should describe a quality problem, not replace technical verification.
Does a CVE ID mean the vulnerability was independently reproduced?
No. It means a CNA assigned and published a record under CVE processes. Independent reproduction, vendor confirmation, environment exposure, and fix verification are additional evidence layers.
Were the six SQLite CVEs real?
JFrog could not reproduce them and found direct contradictions in source, PoCs, and alleged fixes. SQLite lists them as not being SQLite bugs. Their official CVE records were later rejected.
What should happen when a PoC fails?
Preserve the environment and output, check version and configuration assumptions, and request missing evidence. Keep the issue quarantined from automatic remediation until reproducible evidence exists.
Can a rejected CVE still point to a real bug?
Yes. A record can be rejected because the described issue is not a security vulnerability, duplicates another record, targets the wrong product, or contains invalid metadata. Bug existence and CVE validity are related but distinct questions.