Cursor's new agent swarm rebuilt a large part of SQLite from an 835-page manual, and every new model configuration eventually reached 100% on a held-out SQL logic suite. The result is important. It is also easy to misread.
The 100% figure means the generated engine returned the expected results for the tested SQL queries. It does not mean 100% branch coverage, full SQLite compatibility, crash safety, acceptable performance, correct cross-process locking, or production readiness.
The experiment is strongest when read as evidence about the harness, not as a replacement announcement for SQLite.
What Cursor actually tested
Cursor compared its old and new swarms on the same Rust implementation task, with the same model configurations and intended time budget. The agents received the SQLite manual but not SQLite source code, its test suites, the SQLite binary, or internet access. Cursor says the swarm was not told about the held-out sqllogictest, and that researchers reviewed runs for shortcuts after completion.
Four configurations assigned models to planner and worker roles:
- GPT-5.5 for both roles;
- Grok 4.5 for both roles;
- Opus 4.8 as planner and Composer 2.5 as workers;
- Fable 5 as planner and Composer 2.5 as workers.
At the four-hour cutoff, new runs scored between 73% and 85%; old runs ranged from 11% to 77%. Every new configuration later reached 100%, although Cursor did not publish the exact time-to-100 for each one. The old Grok run was stopped before two hours after coordination deteriorated, so it did not consume the full intended budget.
These scores, cost records, conflict counts, and anti-cheating reviews are Cursor-reported. The exact held-out snapshot, grader, full run logs, and replay scripts have not been published.
The real result was less coordination waste
The most persuasive numbers are not the final SQL scores.
In the Grok comparison, the old swarm produced 68,000 commits in its first two hours, about 70 times the new swarm's rate. It accumulated more than 70,000 merge conflicts before being stopped. The new run recorded fewer than 1,000 conflicts over four hours.
The old run's hottest file collected 7,771 conflicts and was touched by 1,173 agents. The hottest file in the new run saw 47. The old design expanded to 54 crates, including three competing SQL packages; the new design stabilized at nine.
Code size tells the same story. Old and new Fable configurations both eventually scored 100%, but used 64,305 and 9,908 lines of engine code respectively. In the Opus hybrid comparison, the old run produced 19,013 lines at 97%, while the new run reached 100% with 4,645.
This is meaningful Harness Engineering evidence. Shared design records, specialized conflict resolution, architecture checks, review lenses, and a task tree reduced thrash. More agent activity was not more progress.
Model assignment changed cost by almost eightfold
Cursor reported new-run costs from $1,339 for the Opus 4.8 planner and Composer 2.5 worker mix to $10,565 for GPT-5.5 in both roles. Workers consumed at least 69% of tokens and more than 90% in most configurations. GPT-5.5 workers alone cost $9,373, while the Composer worker fleet in the Opus hybrid cost $411.
The lesson is not that one mixture is universally eight times cheaper. Prices, task shapes, model behavior, and planner quality can change. The experiment shows why planner and worker economics should be measured separately: most tokens belong to execution, while a smaller amount of expensive planning can change the entire tree.
Do not merge the benchmark with the public repository
Cursor published cursor/minisqlite, but that artifact came from a solo Opus 4.8 run, not the Opus-planner and Composer-worker benchmark configuration. Cursor describes the solo runs as informally graded and says the public artifact had received only an initial review.
The repository independently confirms an impressive amount of work: approximately 200,000 lines of Rust, 14 crates, and 5,650 functions marked #[test]. Its README documents a deliberately small public API.
It also documents boundaries:
- no command-line interface;
- no C API;
- no prepared-statement interface;
- no operating-system file locking or SQLite
-shmprotocol; - concurrent coordination limited to one process;
- no statistics-based query costing;
- no OR-merge and at most one index per table in a plan.
The workspace version is 0.0.0 and publishing is disabled. As of the inspected snapshot, it also did not provide a release, CI workflow, license file, or security policy. These are productization observations, not claims that the code has no technical value.
Most importantly, the public repository cannot reproduce Cursor's private 100% score. Its README says differential testing against real SQLite happens outside the repository.
What sqllogictest proves
SQLite's own documentation is unusually explicit. sqllogictest asks one question: does the database engine compute the correct answer?
It does not evaluate:
- performance or optimal index selection;
- disk and memory use;
- transactional behavior;
- concurrency and locking;
- Unicode and localization behavior.
A 100% score is therefore semantic-correctness evidence for the tested query distribution. It is not a universal database certificate.
This distinction is the database version of the evaluator-interface problem. Search pressure improves what the evaluator measures. Requirements outside that contract remain unoptimized and potentially invisible.
SQLite's release case is a matrix
SQLite does not rely on one suite. Its official test strategy combines four principal harnesses:
- TCL tests with 51,445 distinct cases and millions of executions;
- TH3 with 50,362 distinct cases, about 2.4 million full-coverage instances, and a much larger release soak;
sqllogictestwith millions of queries;dbsqlfuzz, which mutates both SQL and database files.
The wider program covers out-of-memory behavior, I/O failures, power loss during commit, corrupt files, boundary values, leaks, concurrency, multiple platforms, compile-time configurations, regression tests, and fuzzing. TH3 runs against deployed object code and targets branch and MC/DC coverage for the SQLite core.
That breadth exists because database correctness has several independent dimensions. A query can return the right rows during a clean run and still lose data after a power failure.
A three-layer production-readiness gate
Use three separate gates for an AI-generated system.
1. Semantic correctness
- conformance tests and golden results;
- independent hidden cases;
- differential testing against trusted implementations;
- mutation tests to show the suite detects planted faults.
This is where Cursor's held-out SQL suite provides evidence.
2. Engine resilience
- crash recovery and durability;
- OOM, disk-full, short-write, and I/O failure injection;
- corrupt and adversarial inputs;
- multi-thread and multi-process locking;
- fuzzing, sanitizers, leak checks, and race detectors;
- cross-platform and compile-configuration testing;
- performance and worst-case resource limits.
These failures are not weaker versions of a wrong query result. They are different hazards.
3. Productization
- stable and documented APIs;
- migration and compatibility policy;
- prepared statements and required language bindings;
- CI, signed releases, provenance, license, and vulnerability handling;
- observability, rollback, operational ownership, and support expectations.
Production readiness includes the organization that responds when the artifact fails.
How to evaluate the next swarm experiment
Future experiments would become easier to audit if they published:
- the exact task specification and test snapshot hash;
- harness, model, prompt, and tool versions;
- per-run seeds, budgets, logs, and stop reasons;
- every final artifact, not one different solo run;
- independent reproduction instructions;
- semantic, resilience, performance, and interface scores separately;
- failures and manual interventions, not only best results.
Cursor's experiment should not be dismissed because it falls short of SQLite's mature release process. Its contribution is more specific: a redesigned swarm generated much less conflict and much more coherent software across several model mixes. That is strong evidence that harness architecture changes the economics of long-running coding agents.
It is not yet evidence that the generated database should hold production data.
Frequently asked questions
Did Cursor's swarm pass every SQLite test?
No. It reached 100% on Cursor's held-out SQL logic suite. SQLite uses several additional harnesses and many other forms of resilience and release testing.
Is the public minisqlite repository the 4,645-line Opus hybrid result?
No. The public repository came from a solo Opus 4.8 run and is a different artifact.
Does 5,650 tests mean full coverage?
No. It is a count of test functions. Coverage requires separate evidence about statements, branches, conditions, configurations, and fault detection.
What did the experiment prove most clearly?
The new swarm greatly reduced coordination waste. Conflict counts, crate structure, and code size improved even when final SQL scores were similar.