Anthropic's internal maintenance routines opened 388 pull requests and merged 180 after Claude Code Review and human review. The ratio is 46.4%. It is useful evidence that scheduled coding agents can produce mergeable work, but it is not a model success rate. The other 208 pull requests include unknown combinations of waiting, duplication, scope mismatch, CI failure, review rejection, and agent failure.
The correct production metric is a review funnel with an explicit exit reason at every stage.
Evidence reviewed: 19 August 2026. The 388 and 180 counts come from Boris Cherny's public report; workflow behavior comes from Anthropic's current Claude Code documentation.
Reading time: 8 minutes · About 1,500 words
TL;DR
- Claude Code routines are saved autonomous cloud sessions triggered by schedules, APIs, or GitHub events. They are currently a research preview.
- A green run means the session started and exited without an infrastructure error. Anthropic explicitly says it does not mean the assigned task succeeded.
180 merged / 388 opened = 46.4%mixes production output with queue state and rejection causes. It cannot isolate model quality.- Track nine stages from trigger to an unreverted merge, plus time and exit reason at each stage.
- Claude Code Review is best-effort and returns a neutral check by default. A team needs external CI or human approval if review findings should block merging.
What the 46.4% figure contains
Boris Cherny reported that Anthropic had been running daily Claude Code routines across iOS, Android, desktop, web, the CLI, and the Agent SDK. The routines handled tasks such as crash fuzzing, duplicate unification, dead-code removal, and abstraction cleanup.
Over several weeks, they opened 388 pull requests. After automated Claude Code Review and human review, 180 were merged.
That makes 46.4% a valid observed ratio:
merged PRs / all opened PRs at the reporting time
= 180 / 388
= 46.39%
The denominator is not closed pull requests. Public information does not say how many of the remaining 208 were still waiting, superseded, duplicates, rejected, or technically broken. It also does not report first-pass CI, review rounds, post-merge defects, or reverts.
Calling 46.4% a failure rate would be equally misleading. A duplicate maintenance PR may indicate that the routine lacked shared state. A correct PR can wait because reviewers are overloaded. A task can become obsolete after another change lands. Those are workflow losses, not necessarily coding failures.
Run health and task success are different states
Anthropic's Routines documentation states the distinction directly: green status means a session started and exited without an infrastructure error. Blocked network calls, missing connector tools, and task-level failures appear in the run transcript rather than the status indicator.
This creates at least three separate outcomes:
| State | Question answered |
|---|---|
| Infrastructure complete | Did the cloud session start and exit normally? |
| Task complete | Did the routine produce the result defined in its prompt? |
| Business accepted | Did the repository and reviewers accept the change, and did it remain healthy? |
Combining them into one success field hides where to improve the system. Infrastructure errors need platform work. No-op runs need better task discovery. CI failures need stronger local acceptance checks. Review rejection may reflect poor code, duplicate work, architecture disagreement, or changed priorities.
A nine-stage review funnel
Track counts, conversion, queue age, and exit reason at each stage:
- Triggered runs: a schedule, API call, or GitHub event fired.
- Infrastructure-complete runs: the session exited without a platform failure.
- Acceptance-complete runs: repository-specific commands and artifact checks passed.
- Pull requests opened: a reviewable diff with scope and evidence exists.
- CI-passing pull requests: required tests, lint, types, security scans, and policy checks passed.
- Machine-review-cleared pull requests: automated review found no unresolved issue above the team's threshold.
- Human-approved pull requests: a responsible reviewer accepted scope and design.
- Merged pull requests: the change entered the target branch.
- Unreverted changes after 7 or 30 days: the merge survived production feedback.
The first useful visualization is not one headline percentage. It is a funnel with a reason distribution beside every drop.
Classify the 208 before changing the model
Use a mutually exclusive primary exit reason, with optional secondary labels:
| Exit reason | Likely control |
|---|---|
| No actionable work | Improve discovery query and task eligibility |
| Duplicate or superseded | Add repository-level claim and deduplication state |
| Scope mismatch | Tighten target files, owners, and change budget |
| Environment or connector failure | Fix setup, access, or network policy |
| Acceptance command failed | Improve the implementation loop or task selection |
| Machine review rejected | Feed findings back into the same run or next routine |
| Human review rejected: correctness | Improve tests and task context |
| Human review rejected: design | Route to a discussion or design task |
| Waiting for review | Increase reviewer capacity or reduce arrival rate |
| Obsolete before merge | Shorten cycle time and revalidate at review |
| Reverted after merge | Add the escaped failure to the regression suite |
This classification prevents a familiar mistake: upgrading the model when the largest loss actually comes from queue age, stale work, or a missing repository contract.
Research on agent-generated pull requests supports this distinction. One 2026 study of 9,799 human-reviewed agent PRs found that only 35.7% of rejected PRs could clearly be attributed to agent failure; 31.2% involved workflow constraints, while 33.1% lacked an observable decision reason. The absence of a rejection reason is itself a measurement failure.
Acceptance must be executable
A routine prompt should define what completion looks like in the repository. Pair prose with commands and hard limits:
eligible scope: packages/parser/**
required evidence: failing test before, passing test after
required checks: unit, typecheck, lint, security policy
change budget: <= 5 files unless task ledger authorizes more
output: draft PR with summary, risk, test evidence, and rollback note
forbidden: merge, protected-branch push, dependency addition without approval
Claude Code routines run autonomously without an approval prompt during the run. They can use selected repositories, environment variables, network access, and connectors. Actions through connected identities appear as the user. This makes least privilege and an external acceptance contract more important than a long prompt.
Persist cross-run state outside the session. Each trigger creates a fresh session, so use issues, repository files, or another durable task ledger to record claimed work, prior attempts, known failures, and review outcomes. The next run should read that state before proposing new work.
Automated review is a signal, not the merge authority
Anthropic's Code Review uses multiple agents to inspect, verify, deduplicate, and rank findings. The documentation also says review is best-effort, does not automatically retry on failure, and returns a neutral GitHub check. A neutral check does not block branch protection by default.
If the organization wants findings above a severity threshold to block merging, it needs its own CI rule to read the output and fail the gate. Anthropic's product announcement is also explicit that Code Review does not approve a PR; people retain final approval.
This separation creates a clean control model:
- The routine proposes.
- Acceptance commands produce deterministic evidence.
- Automated review finds additional defects.
- CI enforces machine-readable policy.
- A human owns scope and design acceptance.
- Post-merge monitoring decides whether the result survives.
The review bottleneck described in Ramp's Codex workflow becomes measurable rather than anecdotal. The commit-gate pattern in AI Agent Policy Compliance then supplies the enforcement layer.
The dashboard that supports decisions
Report at least:
- first-pass acceptance rate;
- PR-open rate per infrastructure-complete run;
- first-pass CI rate;
- machine-review finding rate by severity;
- human touch rate and review rounds;
- median and p95 queue age;
- merge rate among closed PRs;
- duplicate and obsolete rate;
- 7-day and 30-day revert rate;
- compute and reviewer minutes per unreverted merge.
Segment by routine, repository, task type, model, and prompt version. A global average can hide one excellent dead-code routine and one unsafe abstraction routine.
The objective is not to maximize merge rate. A routine that opens only trivial, guaranteed-to-merge changes can game that number. Optimize valuable, unreverted work per unit of compute and reviewer attention, while keeping risk within the repository's budget.
FAQ
What are Claude Code routines?
They are saved Claude Code cloud configurations containing a prompt, repositories, environment, and optional connectors. Schedules, API calls, or GitHub events can start new autonomous sessions.
Does a green routine status mean the task succeeded?
No. It means the session started and exited without an infrastructure error. Task-level failure must be determined from the transcript and explicit acceptance evidence.
Is Anthropic's 46.4% figure a success rate?
It is the share of all opened PRs that had merged at the reporting time. Without the states and reasons for the other PRs, it cannot isolate model accuracy or final closed-PR acceptance.
Should routines merge pull requests automatically?
Use risk-tiered authority. Low-risk, well-tested changes may qualify after deterministic and policy gates. Architecture, security, dependency, or high-blast-radius changes should retain accountable human approval.
What is the most important funnel metric?
Start with the largest exit category and its queue time. That identifies whether the current bottleneck is task selection, execution, CI, machine review, human review, or post-merge quality.