DeepMind Open-Sources DiffusionGemma: 1000 Tokens/s, 4x Faster Inference

Google DeepMind has open-sourced DiffusionGemma, the first open-source text diffusion LLM under Apache 2.0. It achieves 1000+ tokens/s on H100, 4-5x faster than autoregressive models of comparable size.

💡 What You Will Learn

Google DeepMind has open-sourced DiffusionGemma, the first open-source text diffusion LLM under Apache 2.0. It achieves 1000+ tokens/s on H100, 4-5x faster than autoregressive models of comparable siz

📜 Table of Contents

• DeepMind Open-Sources DiffusionGemma: A Text Diffusion Model Doing 1000+ Tokens Per Second
• How It's Different
• Key Specs
• Who Benefits
• Caveats

DeepMind Open-Sources DiffusionGemma: A Text Diffusion Model Doing 1000+ Tokens Per Second

Google DeepMind just dropped a bomb on the AI world. DiffusionGemma is the first open-source text diffusion model, and it's rewriting the rules on inference speed.

How It's Different

Traditional LLMs are autoregressive — they generate one token at a time, each step depending on the last. Think of it as a typewriter: word by word, one at a time.

DiffusionGemma flips this. It generates 256 tokens in parallel — starts with random noise and iteratively denoises the whole block, similar to how Stable Diffusion generates images. The result? H100: 1000+ tokens/sec. H200: 1288 tokens/sec. Compare that to ~200-300 tokens/sec for similarly sized autoregressive models — that's a 4-5x speedup.

Key Specs

Who Benefits

Local inference users get the most. With 18GB VRAM after quantization, a single RTX 4090 or 5090 can run it comfortably. vLLM added native support on day one via speculative decoding. Unsloth has quantization guides ready, and llama.cpp support is on its way.

Caveats

The trade-off: text quality is slightly below autoregressive models. Diffusion excels at speed over polish. Also, short outputs are slower since each generation produces at least one 256-token block. Apple Silicon users won't see the same speedup due to memory bandwidth limits.

This is local AI inference's inflection point. Apache 2.0, runs on consumer hardware, and faster than any local model before it. The era of cloud-only fast inference just got a serious challenger.