Google’s New AI Tool Can Generate Text Much Quicker

DiffusionGemma-26B-A4B-it is an open multimodal model developed by Google DeepMind that breaks away from the traditional token-by-token paradigm used by most modern chatbots. Rather than predicting one word after another, it produces entire chunks of text and iteratively improves them using a denoising process similar to diffusion-based image synthesis. This allows it to achieve over 1,100 tokens/second using parallel decoding, compared to an autoregressive model’s limited output speed.

It uses a mixture-of-experts approach with 26 billion parameters but only 3.8 billion active parameters per step, activating 8 out of 128 experts per token. It accepts text, images, and videos as inputs, processes up to a 256K context size, supports 35+ languages, has a reasoning mode, and is available under the permissive Apache 2.0 license. While it performs well on various benchmarks (77.6% on MMLU Pro, 70.5% on MATH-Vision), it lags behind the original Gemma 4 model. However, this is an intentional compromise between performance and efficiency.

1Why diffusion for text is a big deal

Unlike autoregressive models, which are inherently serial and rely on previous tokens to produce the next one, limiting their potential speed regardless of the computational resources available, diffusion models break this barrier by processing multiple positions simultaneously and improving them iteratively. While we have been hearing about text diffusion models for some time now, this official release by Google DeepMind on such a large scale, including open weights and a comprehensive multimodal framework, is a clear indication that the technology is moving towards mainstream adoption.

Moreover, the model utilizes sparsity, activating only 3.8 billion parameters out of 26 billion, making it less expensive to deploy and operate than its nominal size.

2Why it matters for you

When developing applications based on language models, latency and cost are often the first two bottlenecks you encounter. With a model capable of generating over 1,100 tokens/second, new possibilities arise, such as real-time document analysis, OCR, and interactive agents without delays. Additionally, having access to open weights under the Apache 2.0 license enables you to deploy the model locally rather than relying on external APIs and paying for each API call.

In conclusion, I would not replace my best reasoning model with this one for critical tasks where accuracy matters most (Gemma 4 still comes out slightly ahead), but DiffusionGemma is definitely worth considering if you need to handle a high volume of requests with low latency requirements. It will allow you to explore the capabilities of a non-autoregressive model in your workflow.