Google debuts DiffusionGemma for faster text generation

According to Google's blog post by Brendan O'Donoghue and Sebastian Flennerhag, Google introduced DiffusionGemma, an experimental open model released under an Apache 2.0 license. The model is a 26B Mixture of Experts (MoE) that, per the post, activates only 3.8B parameters during inference and is designed to run within 18GB VRAM when quantized. Google reports up to 4x faster text generation compared with its autoregressive Gemma models, with throughput claims of 1000+ tokens per second on an NVIDIA H100 and 700+ tokens per second on an NVIDIA GeForce RTX 5090.

According to the blog post, DiffusionGemma moves away from token-by-token autoregressive decoding by using a diffusion-based parallel decoder and a novel diffusion head that generates blocks of text simultaneously. The post describes bi-directional attention, generating 256 tokens in parallel per forward pass so every token can attend to all others, and frames the decode bottleneck as shifted from memory-bandwidth to compute. The blog positions DiffusionGemma as experimental and targeted at researchers and developers exploring interactive, low-latency local workflows.

Editorial analysis: Parallel diffusion-style decoding and MoE parameter activation are emerging approaches to reduce latency for interactive use cases. Companies and open-source projects experimenting with non-autoregressive or partially parallel decoding report similar tradeoffs: materially lower inference latency at the cost of some benchmarked quality relative to strong autoregressive baselines. For practitioners, these patterns matter when choosing model architectures for in-line editing, rapid iteration, or on-device inference where latency and memory constraints dominate.

For practitioners: monitor independent evaluations of output quality across code, dialogue, and long-form generation; check token-level coherence and self-correction behavior under the diffusion decode; and validate throughput claims on your target hardware and quantization setups. Also watch for developer guides and community implementations that document real-world tradeoffs between wall-clock latency and downstream task accuracy.