DiffusionGemma can generate 7.1 times faster than its autoregressive parent in one published setup. It also scores 15.1 points lower on AIME 2026, loses its total-throughput advantage around 32 concurrent requests, and reports generation latency without prefill. The model is fast. Whether the application becomes faster depends on where its bottleneck actually is.
Evidence reviewed: 18 August 2026. DiffusionGemma is an experimental open model; benchmark numbers below retain their published hardware and serving conditions.
Reading time: 8 minutes · About 1,550 words
TL;DR
- The technical report's 7.1x result compares DiffusionGemma at 1,456 tokens per second with Gemma 4 AR at 204 TPS on one H100, FP8, batch size one, 4,096 input tokens, and 1,024 output tokens.
- Against an AR baseline using multi-token prediction, the same report's advantage is 4.8x. Google's launch post says up to 4x; vLLM reports 5-6x versus standard AR and 2.6-3x versus MTP on its setup.
- Speed is purchased partly with capability. On AIME 2026, thinking-mode DiffusionGemma scores 69.1 versus 84.2 for Gemma 4 AR and 88.3 for AR with MTP.
- Diffusion is strongest for a low-concurrency user with long, structurally constrained output. AR batching becomes stronger as concurrent demand rises; the report shows a crossover around 32 requests.
- Deploy both behind a workload router when quality and traffic vary. Route by measured task completion, not peak TPS.
What the 7.1x number measures
DiffusionGemma starts from Gemma 4 26B A4B, a sparse mixture-of-experts model with about 4 billion active parameters. Instead of committing one token after another, it denoises a 256-token canvas with bidirectional attention. A canvas typically converges in far fewer than 256 forward passes.
The technical report averages 19.74 tokens per forward across seven benchmarks. On an H100 in FP8, serving one request with a 4,096-token prompt and 1,024-token output, the authors report:
| Decoder | Output speed | Relative to plain AR |
|---|---|---|
| Gemma 4 AR | 204 TPS | 1.0x |
| Gemma 4 AR with MTP | 303 TPS | 1.5x |
| DiffusionGemma TD | 1,456 TPS | 7.1x |
This is a clean comparison of decoding under one configuration. It is not a universal end-to-end speedup.
Google's launch post uses the more conservative phrase up to 4x faster. vLLM's batch-one benchmark reports 1,288 generation TPS on one H200, about 6x a standard AR baseline and 3x an MTP baseline. On H100 it reports 1,008 TPS, about 5x and 2.6x respectively.
The numbers differ because hardware, kernels, quantization, evaluation prompts, adaptive denoising, and baselines differ. They describe a range of measured systems, not a contradiction.
Why parallel decoding is fast at batch one
Ordinary AR decoding is often memory-bandwidth-bound for a single user. Every next-token step reloads model weights while leaving much of the accelerator's compute capacity idle. Batching improves utilization by generating the next token for many users together.
DiffusionGemma spends that spare compute on one user. It evaluates many positions in a canvas together, trading more computation per forward pass for dramatically fewer passes. The design changes the limiting resource from repeated weight movement toward parallel compute.
That trade is attractive only while spare compute exists. The model is aimed at low-concurrency local or interactive use because an AR server can use the same hardware parallelism to batch many independent users. Google documents this boundary explicitly.
The bottleneck moves with the workload.
The quality bill is visible in the report
The technical report compares text-diffusion mode with the original AR model and an MTP-enhanced AR path. In thinking mode:
| Benchmark | DiffusionGemma TD | Gemma 4 AR | Gemma 4 AR + MTP |
|---|---|---|---|
| AIME 2026 | 69.1 | 84.2 | 88.3 |
| GPQA Diamond | 73.2 | 79.8 | 82.3 |
| LiveCodeBench-v6 | 69.1 | 71.4 | 77.1 |
| HumanEval | 94.5 | 98.2 | 98.8 |
The gap is task-dependent. It is small on some structured coding tasks and large on difficult mathematics. DiffusionGemma also produces fewer total tokens on several reasoning benchmarks. Brevity contributes to speed but limits the longer reasoning traces that often improve accuracy.
The authors identify four causes: the model is warm-started from AR weights rather than pretrained natively for diffusion; diffusion adaptation uses a comparatively short supervised phase; reinforcement learning targets ultra-low latency; and the inherited architecture may be suboptimal for diffusion. Known issues include occasional token stuttering and malformed thought tags in multimodal tasks.
This is a Pareto trade, not a free optimization. Increasing denoising steps can improve precision and latency can be reduced with adaptive stopping. The serving configuration is part of the model.
TPS is not task completion time
The report's detailed latency table excludes prefill. A long prompt, retrieval pipeline, tool call, or verification pass can dominate end-to-end time even when output decoding is extremely fast.
Diffusion also changes the meaning of first-token latency. AR can stream after generating one token. A block-diffusion decoder must refine a canvas before it has a stable block to commit. Fast complete-output generation can coexist with a less immediate first visible token, depending on the streamer and acceptance policy.
For an agent workflow, the useful metric is time to accepted result:
accepted latency = retrieval + prefill + generation + tool waits + verification + retries
If a faster decoder produces more failed answers, the retry and verification terms can erase the TPS gain. This is the same benchmark discipline required when auditing a claimed tokenization bottleneck: measure the stage that limits the complete workflow.
The concurrency crossover
The technical report evaluates total and per-user throughput as concurrency rises. DiffusionGemma leads at low batch sizes. Around 32 concurrent requests, AR models begin to gain a total-throughput advantage.
That number is not a permanent architectural constant. The report notes that DiffusionGemma's kernels and sampling path were not optimized for batch sizes above one. Future implementations may move the crossover. The direction of the tradeoff is more stable than the exact threshold:
- Low concurrency leaves compute idle, which diffusion can use for one request.
- High concurrency gives AR enough independent sequences to use that compute through batching.
- Long prompts can make prefill and attention more important than decode.
- Large canvases and more denoising steps increase compute and memory pressure.
A cloud service should therefore benchmark a traffic distribution, not one batch size.
Four workload decisions
Interactive local generation
DiffusionGemma is a strong candidate when one user owns the accelerator, expects a substantial answer, and values completion latency more than immediate token streaming. Local coding assistance, structured extraction, and offline document transformation fit the intended regime.
High-QPS cloud chat
AR remains the safer default when many requests can be continuously batched. Measure p50 and p95 latency, total completed requests per second, and cost per accepted answer across the actual arrival pattern.
Hard reasoning
Use AR or a stronger reasoning model when answer quality dominates latency. The AIME gap is too large to hide behind TPS. A hybrid can let diffusion draft or handle constrained sections, then route uncertain or high-stakes tasks to AR.
Structured transformation
Diffusion has a structural advantage when output positions are heavily determined by the input. The report shows JSON extraction converging in two denoising steps and localized Python repair in three. These are better targets than open-ended long reasoning.
A deployment matrix that can survive model updates
Freeze a representative task set and compare complete systems. Record at least:
| Dimension | Minimum measurement |
|---|---|
| Quality | Task pass rate, schema validity, factual or test accuracy |
| Latency | Time to first useful output and time to accepted result |
| Traffic | p50/p95 at realistic arrival rates and concurrency |
| Efficiency | GPU-hours and cost per accepted result |
| Stability | Stuttering, malformed output, timeout, and retry rate |
| Configuration | Hardware, precision, engine version, canvas, steps, stopping rule |
Then implement an explicit router:
- Send low-concurrency, constrained, latency-sensitive tasks to diffusion.
- Send difficult reasoning, high-stakes decisions, and quality-sensitive code to AR.
- Send high-QPS traffic to the decoder that wins under measured batching.
- Re-evaluate the boundary after every model, kernel, or traffic change.
The correct fallback is driven by task risk and current bottleneck, not model identity.
FAQ
Is DiffusionGemma really 7x faster?
It is 7.1x faster than plain Gemma 4 AR in the technical report's batch-one H100 setup. The advantage is 4.8x against MTP AR there, while other official implementations report different multiples.
Why does Google's launch page say 4x?
It uses a broader, more conservative launch claim. Benchmark results vary with hardware, precision, serving engine, prompts, adaptive steps, and baseline.
Does DiffusionGemma have lower reasoning quality?
On the published AIME 2026 comparison, yes: 69.1 versus 84.2 for the AR parent. The gap is smaller on some coding and structured tasks, so quality must be measured per workload.
Is diffusion decoding always lower latency?
No. It strongly reduces batch-one generation time, but prefill, first committed output, tool calls, verification, retries, and concurrency can dominate end-to-end latency.
When should a system fall back to autoregression?
Use AR when hard reasoning or high-stakes accuracy matters, when high concurrency lets batching win, or when the measured cost per accepted result is lower despite slower raw decoding.