Artificial Intelligence answers 15 times faster with no compromise on quality.

Up to 15x speedup. That’s how much faster DFlash made a 120B parameter model on NVIDIA’s newest generation of hardware, called Blackwell. Blackwell is currently deployed in many of the world’s largest cloud data centers.

1One block at a time instead of one word at a time

AI generation is usually done sequentially, much like typing a note character by character. When generating responses, the model produces one token at a time, then moves onto the next. As you might imagine, this leads to high latency. With DFlash, a small and efficient helper model generates an entire block of tokens at once, like printing out an entire word. Then the primary model iterates over the block and either keeps or rejects the tokens. The rejection process happens in parallel.

The key insight behind DFlash is that the helper model receives direct access to the internal states of the primary model. This allows the helper model to generate tokens similar to what the primary model would produce. As a result, more of the block is kept, requiring less rejection, and it happens quicker.

2The numbers

According to the research paper, DFlash achieves an average speedup of 4.86x, with a maximum lossless speedup of 6.08x on Qwen3-8B. We evaluated DFlash on a 120B model running on Blackwell and found up to 15x throughput while maintaining the same interactivity. It also surpassed the state of the art in this area by approximately 2.5x. Additionally, we saw substantial speedups on other model architectures such as Qwen3 (8B and 27B), LLaMA 3.1, and Gemma 4 31B.

3What this means if you're building with AI

For those who host their own open AI models (e.g., for a product, internal tool, or code generation service), DFlash allows you to support significantly more users with the same amount of hardware. For example, a 5x speedup would correspond to handling roughly 5x more requests/minute per server. This makes it financially viable to build a product around open models instead of paying for API calls.

DFlash is completely open source. We released checkpoints on Hugging Face and integrated it with vLLM, Transformers, and TensorRT-LLM, which are the three most popular frameworks for hosting open AI models. There are no proprietary dependencies and no licensing requirements. This isn’t an announcement about a benchmark with a 12-month waitlist; it’s a release of actual working code that you can start using right away.