DeepMind Open-Sources DiffusionGemma: 1000 Tokens/s, 4x Faster Inference

Google DeepMind released DiffusionGemma, the first open-source text diffusion LLM under Apache 2.0 with vLLM support shipping same day. Instead of autoregressive token-by-token generation, it produces blocks of 256 tokens in parallel, iteratively denoising random placeholders into coherent text. Key specs: 26B total params (3.8B activated MoE), 256K context, 700+ tokens/s on RTX 5090, 18GB VRAM quantized. Community reactions are split: proponents call it a local inference revolution, critics note quality lags behind autoregressive models and short outputs are slower.

DiffusionGemma won't replace autoregressive models — the quality gap remains for high-precision tasks. But local inference has hit an inflection point. For regular users: fast local inference without cloud APIs. For developers: vLLM support means RTX 4090 can now deliver cloud-grade speeds. Google has opened the text diffusion path under Apache 2.0 — expect more teams to follow in 6-12 months.