AI-assisted runtime rewrites can now compress years of work into weeks or months. Their headline speedups still mix three different causes: language and runtime changes, architecture changes, and verification-driven optimization. This article uses GitHub Copilot, Bun, and Prime Agent to show how to separate them before deciding that a rewrite worked.
Reading time: 8 minutes | Words: about 1,600
TL;DR
- Final before-and-after numbers do not isolate what Rust, architecture, or later tuning contributed.
- GitHub and Prime expose useful intermediate baselines that make partial attribution possible.
- A compiler is a high-frequency rejection mechanism, not a proof of behavioral equivalence.
- Track five ledgers: runtime, architecture, state and cache, verification, and total human plus infrastructure cost.
A rewrite benchmark is a causal claim
When a TypeScript runtime becomes Rust and gets 14 times faster, the tempting story is simple: Rust caused the gain. Real rewrites change many variables together.
The old system may spawn a process for every client. The new system may run in-process. Polling may become event-driven. Startup work may move off the critical path. A new benchmark harness may expose problems nobody had measured. The team may also simplify dependencies, protocols, state ownership, and packaging.
The right question is not whether the final system is faster. It is which bottleneck moved, and why.
Use three primary buckets:
| Gain | Typical mechanisms | Best evidence |
|---|---|---|
| Language and runtime | Native startup, no V8 or GC overhead, static linking, ownership checks | Same architecture, different implementation |
| Architecture | In-process embedding, fewer boundaries, event-driven waits, modular state | Same implementation before and after architecture change |
| Verification and tuning | Compiler feedback, differential tests, benchmarks, fuzzing, dogfood | Sequential experiment log against a frozen baseline |
Without intermediate states, attribution remains a hypothesis.
Case 1: GitHub Copilot removed a process boundary
GitHub rewrote its Copilot agent runtime from TypeScript on Node.js and V8 into more than 800,000 lines of production Rust. The port took about 14.5 weeks and landed through 128 port pull requests while the main branch shipped 135 releases.
The old SDK launched a CLI process for each client. That process hosted Node and V8, communicated through JSON-RPC, and added roughly 100 MB of minimum working-set overhead per client. The Rust design offered both an out-of-process mode and an in-process C ABI.
That intermediate state matters. In GitHub's structured table, a client, session, and one-turn runtime path fell from 5.25 seconds in TypeScript to 1.33 seconds in Rust out-of-process, then to 292 milliseconds in-process. The first transition mixes language, runtime, and concurrent code changes. The second, roughly 4.6 times improvement, more directly exposes the value of removing process and IPC boundaries.
The benchmark excludes model inference and network latency. It measures the runtime path, not end-to-end coding-task completion. GitHub's page also contains a conflicting roughly 55 millisecond figure in another presentation of the result. The safest practice is to use the structured comparison and disclose the inconsistency rather than select the larger headline.
The rewrite's strongest engineering choice was incremental replacement. Existing end-to-end tests ran against each Rust slice, every pull request remained shippable, and real consumers provided staged production feedback. The rewrite was therefore also a change to the verification architecture.
Case 2: Bun kept performance gains modest and measurable
Bun moved a large Zig codebase to Rust with one engineer directing a large Claude workflow. It reached an all-platform green test suite in 11 days, but the article is unusually clear about false starts, compiler failures, adversarial review, later security review, and continued fuzzing.
The measured application and HTTP improvements were mostly 2% to 5%, with a 10% Linux startup improvement for a downstream Claude Code release. Bun attributes much of the smaller performance gain to cross-language link-time optimization, not to a universal Rust advantage.
Other improvements had different causes:
- a large repeated-build memory leak was fixed through explicit ownership and cleanup;
- binary-size reduction combined the rewrite with ICU trimming and identical-code folding;
- stronger safety came from the borrow checker, Miri, sanitizers, CI, and continuous parser fuzzing.
The initial translation still produced about 16,000 compiler errors. All code received two adversarial reviews, and the final suite covered more than 57,000 tests on each major platform. Yet the team still recorded known regressions. Compile-green removed mechanical inconsistency; it did not prove behavioral parity.
Case 3: Prime Agent publishes the missing middle state
Prime Agent provides the cleanest attribution lesson. Its public table contains three stages: TypeScript, Rust before performance hill-climbing, and Rust after three days of benchmark-driven optimization.
| Metric | TypeScript | Rust before tuning | Rust after tuning |
|---|---|---|---|
| Cold input-ready latency | 736.1 ms | 293.3 ms | 51.9 ms |
| Warm input-ready latency | 552.3 ms | 276.7 ms | 41.4 ms |
| 10 MiB session memory | 1,130.0 MB | 554.9 MB | 237.3 MB |
| Installed size | 172.1 MB | 206.7 MB | 59.6 MB |
| Agent-view switch | 78.0 ms | 118.8 ms | 12.8 ms |
The first migration stage improved cold startup by about 2.5 times and memory by about 2 times. It also made installed size and view switching worse. The later hill-climb added the larger gains by moving work off startup and render paths, replacing polling with event-driven waits, and releasing memory earlier.
Therefore, the final roughly 14 times result is a rewrite plus architecture plus explicit optimization result. Prime also warns that its external harness comparison uses a custom benchmark and lacks a common cross-tool standard.
The compiler is a verifier with a bounded jurisdiction
Rust creates a valuable machine-readable feedback loop. GitHub reports that most compiler diagnostics were ordinary wiring problems such as names, missing methods, type mismatches, and trait bounds. Ownership, borrowing, and lifetime errors were a small share of categorized diagnostics.
That is still useful. Agents can turn compiler output into a work queue, retry rapidly, and reject many invalid changes before human review. But known regressions in all three projects could compile. The remaining failures involved contracts, ordering, state lifecycle, missing behavior, interop, incorrect test oracles, and unexercised paths.
Microsoft Research's RustAssistant study reinforces the boundary. With five GPT-4 completions, 134 of 182 historical Rust commits containing compilation errors were made compilable. A manual audit found that some fixes changed runtime behavior. Compilation success was the inner loop, not the acceptance test.
A complete rewrite oracle needs multiple layers:
- compiler and static analysis for local invalidity;
- unit and cross-platform tests for known behavior;
- differential tests against the old implementation;
- protocol, transcript, and state-persistence checks;
- production canaries, dogfood, telemetry, and rollback.
Keep five ledgers
Use five separate ledgers before approving a rewrite:
| Ledger | What to record |
|---|---|
| Runtime | Startup, steady-state latency, CPU, memory, binary and install size |
| Architecture | Process boundaries, IPC, dependency direction, state ownership, failure isolation |
| State and cache | Build cache, prompt cache, repository orientation, checkpoint and replay costs |
| Verification | Compiler rejects, tests, differential coverage, fuzzing, escapes, rollback events |
| Total cost | Tokens, compute, CI, human direction, review, follow-up work, and release tail |
This prevents two common accounting errors. Cached prompt tokens explain the economics of the migration process, not the runtime speed of the shipped product. A fast initial translation also does not include the time required to reach release quality.
A practical rewrite gate
Before starting, freeze the old implementation as a behavioral oracle and define representative workloads. During the rewrite, preserve intermediate baselines instead of comparing only the oldest and newest systems. Ship small slices, attach verification evidence to every slice, and measure the bottleneck again after each meaningful improvement.
Approve the rewrite only when it satisfies all four conditions:
- the target workload improves, not just a microbenchmark;
- parity evidence covers critical states and failure paths;
- total migration and maintenance cost is recorded;
- the new architecture removes a constraint that matters in production.
Frequently asked questions
Does Rust automatically make AI-generated code correct?
No. Rust rejects many mechanical and memory-safety errors early. Behavioral contracts, ordering, omissions, interoperability, and bad test oracles still require independent verification.
How much of Prime Agent's 14x gain came from Rust?
The public data cannot isolate pure language causality. The pre-tuning Rust stage improved cold startup by about 2.5 times; later architecture and benchmark-driven changes added roughly another 5.6 times.
Can GitHub's runtime speedup predict complete coding-task speed?
No. Its benchmark excludes inference and network time. It is evidence about startup, session, event, persistence, and teardown overhead.
Did Bun produce one million lines of production Rust in 11 days?
The million-line figure refers to landed diff volume, not final production lines. Bun's article describes roughly 780,000 Rust lines, and release-quality work continued after the initial all-platform test milestone.
What is the most important rewrite asset?
A behavior oracle independent of the new implementation: old-system outputs, protected differential tests, cross-platform CI, protocol traces, and production canaries.
Next action: If a rewrite proposal offers one before-and-after speedup, ask for the missing middle state. One additional baseline often reveals whether the gain came from the language, a changed boundary, or the optimization campaign that followed.
References
- GitHub: Migrating the Copilot runtime to Rust
- Bun: Rewriting Bun in Rust
- Prime Intellect: Rewriting Prime Agent in Rust
- OpenAI Codex discussion: Moving from TypeScript to Rust
- Microsoft Research: RustAssistant
- Multi-SWE-bench
- Related: Programming languages are agent harnesses
- Related: Why 100% tests are not production