Nemotron-Labs Diffusion

Nemotron-Labs Diffusion is a family of downloadable language models from NVIDIA (May 2026) that can write text faster than usual. A normal chatbot model writes one word piece at a time; these models can also fill in a block of text at once and then correct it. For a business that runs its own model, say on an office server, that can mean faster answers from the same machine.

What is in the family?

On 19 May 2026 NVIDIA released text models of 3, 8 and 14 billion parameters, each as a base and an instruction-tuned version, plus an 8-billion-parameter vision-language model (GitHub). The text models fall under the NVIDIA Nemotron Open Model License, which states that they are commercially usable; the vision-language model comes under NVIDIA's Source Code License (launch post). According to the technical report, NVIDIA started from the Ministral 3 base models of Mistral AI in the same sizes, trained them further on 1 trillion tokens in the usual way and then on 300 billion with a combined objective, and fine-tuned the instruct versions on 45 billion.

How does it work?

A model like GPT writes autoregressively: each new token needs a full pass through the model, and a written token is final. Diffusion, known from image generators, starts from noise and refines it in steps. For text, diffusion language models such as LLaDA (2025) hide tokens behind a mask and learn to fill them in. Nemotron-Labs Diffusion can do both and switches by changing its attention pattern (model card). That gives three modes:

• Autoregressive: left to right, one token per pass. NVIDIA prefers this mode when a server handles many requests at once.
• Diffusion: the model fills in a block of text, 32 tokens in NVIDIA's examples. The block starts fully masked, and at each step the model commits the tokens it is confident enough about. Within a block the tokens see each other; between blocks the model works left to right, so finished blocks can reuse the KV cache.
• Self-speculation: the diffusion mode drafts several tokens in one pass and the autoregressive mode checks them in a second pass; the longest part that matches is kept. This is speculative decoding in which the model writes its own drafts. According to NVIDIA, it works best when few users are served at once.

Training combines both objectives: the usual next-token loss plus 0.3 times a diffusion loss on masked blocks. The approach builds on NVIDIA's Efficient-DLM (December 2025), which turns a trained autoregressive model into a diffusion model.

How much faster is it?

NVIDIA measures part of the speed in tokens per forward pass (TPF): the number of tokens per pass through the model, a measure that does not depend on the hardware. For the 8B model, the launch post reports 2.6 times the TPF of autoregressive models in diffusion mode and 6 to 6.4 times in self-speculation, with comparable accuracy, and an average accuracy 1.2% higher than that of Qwen3 8B. On real hardware the gain is smaller. With one user at a time, self-speculation produced 850 tokens per second on a GB200 GPU against 253 in autoregressive mode, and 112 against 41.8 on NVIDIA's DGX Spark desktop computer with 4-bit weights (model card).

Where does it go wrong?

• The gain depends on the load. With many simultaneous requests the GPU is already busy, and according to the technical report the autoregressive mode is then the better choice.
• All speed and accuracy figures come from NVIDIA itself, measured on NVIDIA hardware.
• In diffusion mode a threshold decides when a token counts as certain; a lower threshold commits more tokens per step and makes more errors. Self-speculation has no such trade-off: at temperature 0 its output is identical to that of the autoregressive mode, according to the launch post.
• The software was young at launch. The models need Hugging Face Transformers 5 with custom model code, and serving with SGLang relied on a pull request that was still open in September 2026.