Diffusion Gemma Showdown

Why I Built This

Not every project here is a polished product. This one isn't, and it isn't trying to be.

Google released DiffusionGemma, a text diffusion language model, in June 2026. I kept reading claims about how it generates text completely differently from a normal model, and I didn't want to just take that on faith. So instead of another read and move on session, I built the smallest possible thing that would let me see the difference for myself: put the same broken paragraph in front of a regular model and a diffusion model, and watch what each one actually does.

This was built fast, broke often, and taught me more from the debugging than from the parts that worked on the first try. If you're looking for a production app, this isn't it. MemoryVault or Codebase Explainer are closer to that. This one is a hands on learning experiment, and I'm writing it up that way.

What It Does

Paste a paragraph that has a factual mistake planted in it (I used one about the Eiffel Tower, with the city, the year it was built, and its height all wrong). Click one button, and two things happen live, side by side:

• Gemma 4 (the regular, word by word model) streams its answer in, fixing the mistakes.
• DiffusionGemma shows its whole block and progressively refines it in place.

Both panels report a real number underneath: the percentage of words each model actually changed to fix the same mistakes, not a guess, an actual word by word diff against the original.

Architecture

Gemma 4's answer comes from a plain API call to Google. Nothing is loaded or run locally for that side. DiffusionGemma is the opposite: the actual 26 billion parameter model is loaded and run directly inside the app, borrowing a temporary free GPU only for the seconds it's actually generating.

Tech Stack

| Layer | Technology | |---|---| | App / UI | Gradio, hosted on Hugging Face | | Gemma 4 access | google-genai SDK, calling the Gemini API | | DiffusionGemma access | transformers (DiffusionGemmaForBlockDiffusion), run locally | | Diffing | Python's built in difflib | | Hosting | Hugging Face Spaces, free tier |

What I Actually Found

Going in, my assumption was that DiffusionGemma would visibly touch fewer words than Gemma 4, since it isn't forced to regenerate the whole paragraph the way a strictly left to right model is. That's not quite what happened.

With a plain instruction (fix the mistakes, don't change anything else) rather than an instruction that locks specific word positions, both models landed on nearly the same percentage of words changed. The full reasoning is in the blog post. Short version: the only touches what's broken advantage is real, but it requires masking the exact positions you want preserved, which a plain prompt doesn't give you. What is real and visible without any of that is the mechanism itself: one model typing in order, the other filling in and refining a whole block at once, live on screen.

What I Learned

1. A model's thinking can leak into the text you're measuring. DiffusionGemma's raw output included the entire prompt echoed back, plus internal role markers like thought, unless explicitly told to skip the prompt and unless I stripped stray markers myself. My first working demo was accidentally measuring my own instructions as if the model had rewritten them.

2. Two things running in one process compete for the same CPU, even with a GPU involved. A lightweight network call and a heavy GPU bound generation running at the same time in the same Python process can starve each other for CPU time, making the lightweight one look hung when it's actually just waiting its turn. Running them one after another, deliberately, turned out to be more reliable than trying to force real concurrency.

3. Free shared GPU access doesn't reset all at once. It comes back gradually, tied to when each individual second was actually used. That changed how I paced testing throughout the day.

4. Streaming libraries aren't always built to be iterated. DiffusionGemma's official streaming class is built to print colored text to a terminal, not to hand data back to calling code. Using it required subclassing it and intercepting its actual callback methods rather than treating it like a normal Python iterator.

5. A quick fix can hide the real bug. More than once, patching the symptom I could see (a hang, a wrong number) turned out to be treating the wrong cause, and the real fix only showed up once I pulled the actual server logs instead of guessing from what the page displayed.

The full commit history on GitHub has the real trail, mistakes included.