NVIDIA's Sigma Writes in Parallel Blocks and Runs About 3.5x Faster Than Normal Decoding

A green circuit board with gold traces on a black background

Photo: Hippopx (public domain). Illustrative image.

What happened. A research team from NVIDIA, Cornell University and Georgia Tech posted an updated version of their paper “Large Language Continuous Diffusion Models” to arXiv on 12 November 2026. It details Nemotron-Labs-Sigma, a family of 3-billion and 8-billion-parameter “continuous diffusion” language models that the authors describe as the first large-scale continuous diffusion language model — and claim it delivers roughly 3.5 times the throughput of standard autoregressive decoding while keeping the model’s answers.

How Sigma writes differently. For most of the generative AI era, language models have written the way a person types: one token at a time, each word waiting on the one before it. That sequential habit is a big reason inference is slow and expensive at scale. Diffusion models became famous in image generation, where a system starts from random noise and refines it, step by step, into a picture. Applying the same idea to language is harder, because words are discrete symbols, not smooth pixel values. Most text diffusion models to date have worked by masking tokens and filling them back in, which the authors argue creates a jagged, high-dimensional space that is hard to steer. Sigma takes a different route: each token is represented as a point in a compact 16-dimensional embedding space, the model adds Gaussian noise to a block of these embeddings and learns to clean it up, guided by the text that came before. When a block is finished, it is converted back into words and added to the context, and the next block begins. To keep training costs manageable, the team warm-started Sigma from existing pretrained autoregressive model weights rather than training from scratch.

The numbers behind the speed claims. The headline efficiency results come from two techniques. The first is distillation, in which a slower “teacher” model trains a “student” to reach similar answers in far fewer refinement steps. The authors report roughly 8x acceleration on math and 4x on coding, equivalent to producing 8 and 4 tokens per forward pass, with what they describe as negligible accuracy loss. The detailed figures show where that holds and where it slips: the distilled 3B model running just 8 denoising steps scored 84.62% on the GSM8K math benchmark, against 85.57% for its full teacher run at 256 steps, and 56.40% versus 57.26% on HumanEval coding. But on the MBPP coding benchmark, the gap widened to 49.04% versus 58.00% — a real cost the authors acknowledge. The second technique is self-speculative decoding: Sigma’s diffusion side drafts a 32-token block in one pass, and the same model’s autoregressive side checks the draft, accepting the longest stretch that matches what it would have written itself. Because the final output is that model’s own greedy autoregressive answer, the speedup does not change the result. Measured on a single NVIDIA GB200 using SGLang with one request at a time, the 8B model averaged 3.46x throughput over its own autoregressive decoding across GSM8K, HumanEval and MBPP, compared with 3.04x for NVIDIA’s earlier masked-diffusion model, Nemotron-Labs-Diffusion.

Accuracy: a modest edge, not a knockout. After pretraining, the regularised Sigma-8B averaged 71.16% across six math and coding benchmarks, versus 70.22% for the matched-size Nemotron-Labs-Diffusion-8B, while trailing it on MBPP and MBPP+. Where Sigma stands out is when many answers are sampled: an MBPP pass@256 score of 91.8%, compared with 87.6% for Nemotron-Labs-Diffusion, a sign the model can find more distinct correct solutions when given more tries.

The caveats matter. These are 3B and 8B research models, not frontier-scale systems. The throughput figures were measured under a single-request serving setup, which is not the same as a busy production cluster handling many users at once. And reinforcement-learning post-training, a standard step in making today’s leading models useful, is left as future work. The paper says the model weights are publicly available.

Why it matters. Inference, not training, is where most companies feel AI costs every month. A technique that produces several tokens per forward pass without changing the answer goes straight at that cost line — and it is a different lever from the usual ones, smaller models and better chips. If speedups of this kind survive at larger scales and under real traffic, the same hardware could serve more customers, or the same workload could run on less hardware. Strategically, the work fits a pattern for NVIDIA: publishing open research and weights that make its GPUs more productive keeps developers building on its stack. The practical move today is not to switch architectures, but to track whether diffusion-style decoding lands in mainstream serving engines. For a different kind of research breakthrough this week, read our report on DeepMind’s AlphaGenome 2, and for the basics, our plain-English guide to large language models.

Sources

  • NVIDIA
  • diffusion models
  • Sigma
  • inference
  • arXiv

Related articles