DiffusionGemma: Google Converts Gemma 4 Into a Diffusion Model for Under 10% of Training Cost

Fine tuning 26B parameter Gemma 4 model to diffusion model with <10% of training budget. DiffusionGemma has a batch size of 256 tokens (compared to single token LLM output), runs at ~1,500 tokens/sec on Nvidia H100, and has slightly worse quality than the original model. DiffusionGemma improves by 10 points on reasoning tasks and can solve 85% of Sudoku puzzles (0% for the baseline).

Previously, diffusion text models could only be trained from scratch, making them inaccessible to anyone but the largest companies. However, if we can efficiently fine tune any model to diffusion, then we will have access to a wider range of cheaper specialized versions of diffusion models. The balance of power in the generation efficiency arms race has shifted!