DiffusionGemma: The AI That Writes Every Word at Once, Not One at a Time

Every AI model you have ever talked to writes the same way. Left to right. One word, then the next, then the next. It never goes back.

In June 2026, Google released a model that does not do that. It writes an entire block of text, all 256 words of it, at the same time, then keeps refining the whole thing until it settles into something readable. It is called DiffusionGemma, and it borrows its core idea from the same technology that makes AI image generators work.

I wanted to understand what that actually means, not just read about it, so I built a small side by side demo to watch both approaches happen live. More on that at the end. First, here is what DiffusionGemma actually is. No jargon, just the real mechanism.

The Way Every AI Model Writes, Until Now

Picture someone typing on an old typewriter. Once a letter hits the page, it is permanent. You cannot go back and fix an earlier letter without retyping everything after it.

That is how a normal AI model, including Google's own regular Gemma 4, writes text. It is called autoregressive generation: the model predicts one word, commits to it, then uses everything written so far, including that word, to predict the next one. Every word depends on every word that came before it, generated strictly in order.

This is why AI models sometimes feel slow on long answers. They are doing this prediction step, one word at a time, however many hundred words you asked for.

What DiffusionGemma Does Differently

Now picture a whiteboard covered in 256 blank word shaped slots, all filled in with random gibberish. DiffusionGemma looks at that entire whiteboard at once and asks which of these words look wrong. It erases the ones it is unsure about, guesses again, and repeats, over and over, until the whole board reads as a coherent block of text.

This is called diffusion, and it is the same underlying idea used by AI image generators like Stable Diffusion, which start with random static and gradually clear it into a picture. DiffusionGemma applies that same process to text instead of pixels.

Watch what that actually looks like, stage by stage. Orange means the model has no idea yet. Yellow means it is getting close. Green means the word is locked in for good.

Here is the loop those four snapshots came from, step by step:

1. Start with a block of 256 tokens (roughly 256 word pieces) filled with random placeholder content.
2. In each pass, the model looks at all 256 positions at the same time. Every word can see every other word in the block at once, not just the words before it.
3. Positions the model is confident about get locked in for good. Positions it is still unsure about get reset to random noise and tried again.
4. This repeats, up to 48 times, until the whole block settles into finished text.

Once a 256 word block is finished, DiffusionGemma appends it to what is already written and starts a fresh block for the next chunk. So a long answer is still built block by block. It is inside each 256 word block that everything happens together, not one word after another.

The Real Difference, Side by Side

Why This Can Actually Be Faster

Here is the technical reason this matters, and it is genuinely well explained by Google, NVIDIA, and the team behind the vLLM inference engine. All three independently describe it the same way.

Normal one word at a time generation is what is called memory bound. For every single word, the model has to pull its entire set of weights through the GPU's memory, do a small amount of actual math, then repeat for the next word. Most of the time is spent moving data around, not computing.

Diffusion flips that. Pulling the same weights through memory once and using them to work on 256 words in parallel turns the job into a compute bound task. The GPU spends its time actually calculating instead of waiting on memory. That is exactly the kind of workload modern GPUs are built to be fast at.

The measured result: Google reports over 1,000 tokens per second on a single NVIDIA H100 GPU, and the vLLM team independently benchmarked up to 1,288 tokens per second on an H200. Both cite roughly a 4x speedup over standard one word at a time generation.

The Catch: Speed Is Not Free

Here is the part a lot of the excitement around this launch leaves out, and it is worth saying plainly. DiffusionGemma is not simply a faster version of Gemma 4. It trades some accuracy for that speed.

Google's own release materials say this directly: standard, autoregressive Gemma 4 remains their recommendation for high quality production output, while DiffusionGemma is positioned for speed critical, interactive local workflows. Independent testing backed this up. On several benchmarks, the 26 billion parameter DiffusionGemma actually scored a little below Google's own smaller 12 billion parameter regular Gemma 4 model.

There is a technical reason for the accuracy gap too. DiffusionGemma was not trained as a diffusion model from scratch. According to Google's own technical report, it was created by taking the existing autoregressive Gemma 4 and retraining it for diffusion style generation using less than 10% of the original model's full training budget. A fast retrofit, not a from scratch build. That shortcut is part of what made the speed gains possible on such a short timeline, and part of why quality took a small hit.

It is also explicitly labeled experimental, not production software. A few other things worth knowing if you go looking:

• In heavy cloud environments serving many users at once, the speed advantage can flip into a cost disadvantage. The memory bound bottleneck diffusion solves mostly shows up when serving one user at a time, not many at once.
• Independent testers have reported it occasionally getting stuck in short repetition loops, an artifact attributed to the shortened training process.
• Google itself says, in its own words, to keep using regular Gemma 4 whenever output quality matters more than speed.

None of this makes DiffusionGemma uninteresting. It makes it an honest trade off, which is a more useful thing to understand than a headline number.

Where This Actually Shines

The speed is not the only thing that changes. The fact that every word in a block can see every other word (instead of only the words before it) opens up tasks that are awkward for normal left to right generation:

• Inline editing and code infilling: fixing a bug in the middle of a function without regenerating everything after it.
• Structured content, like Sudoku puzzles, where a normal model struggles because later cells constrain earlier ones, and a left to right model cannot see a constraint that comes after the word it is currently writing.
• Working with sequences that do not have a natural left to right order, like mathematical graphs or amino acid sequences.

I Tested This Myself

Reading about the mechanism is one thing. I wanted to actually watch it happen, side by side against regular Gemma 4, on the same task, so I built Diffusion Gemma Showdown: a small live demo where you paste a paragraph with a planted mistake, and both models try to fix it in front of you, at the same time.

The honest result surprised me a little, and it connects directly to everything above. When both models are just given a plain instruction (fix the error, change nothing else) rather than being told exactly which word is wrong, DiffusionGemma does not automatically touch fewer words than Gemma 4. The two landed on nearly identical results in my testing. The theoretical only touches what is broken advantage is real, but it specifically requires telling the model exactly which positions are locked and which are open for editing, a level of control a plain prompt does not give you. What is real and visible without any of that: watching the two actually generate is completely different. One typing in order, the other filling in and refining the whole block at once.

Full write up, the code, and what actually broke along the way are on the project page here.

---

DiffusionGemma is not going to replace how most AI models write text. Even Google says so. But it is a genuinely different mechanism, running in production adjacent form today, not just a research paper. Understanding how it writes, not just that it is faster, is what made this worth digging into for me.

Everything I am building is at ashishmehrotra.com. If you are exploring the same space, reach out.