Denoising blocks, not tokens
Denoising Blocks, Not Tokens: Efficient Compressed Continuous Diffusion with Branching Token Realization
- Identifier
- arXiv:2610.09311
- Published
- Oct 7, 2026
- Authors
- 6
- Type
- Paper
- Area
- Enterprise-optimized models
In one paragraph
Diffusion language models keep one state per token, so every denoising step grows with the output length. Branching Latent Diffusion compresses 1,024 tokens into 64 block latents and decodes each block with a small local branch, all in parallel. Generation FLOPs drop more than 80×.
- fewer generation FLOPs than the ELF-L diffusion baseline
- >80×
- fewer generation FLOPs than the ELF-L diffusion baseline
- higher throughput than ELF-L and than an autoregressive baseline
- >6×
- higher throughput than ELF-L and than an autoregressive baseline
- lower latency than the autoregressive baseline
- >4×
- lower latency than the autoregressive baseline
- compression: 1,024 tokens into 64 block latents
- 16×
- compression: 1,024 tokens into 64 block latents
FIG. 02: Mechanism
How it works
Compress the sequence
A fusion encoder splits a 1,024-token sequence into 64 contiguous spans and pools each into one 512-dimensional block latent. Diffusion now runs over 64 states instead of 1,024.
Generate latents in groups
A latent prior produces the 64 latents in groups of 8. Each group is denoised with rectified flow, conditioned on the groups before it, which keeps strongly compressed latents coherent.
Branch back to tokens
Each latent gets a small causal decoder branch that writes its span of about 16 tokens while attending to all 64 latents. All 64 branches run at the same time.
Pay per block, not per token
Each denoising step touches a handful of latents instead of every token, so generation needs about 1.1 TFLOPs per sequence against 96.4 for the ELF-L diffusion baseline.
FIG. 03: Interactive figure
Denoise 64 latents, not 1,024 tokens.
Press generate to run both models side by side, then change the group size and step count to see what each costs.
Loading interactive demo…
FIG. 04: Notes
Summary
Diffusion language models promise parallel generation, but they keep one state per token, so the cost of each denoising step scales with sequence length. Branching Latent Diffusion (BLD) denoises blocks instead:
- Compress. A 1,024-token sequence is represented as 64 block latents, a 16× reduction, so diffusion runs over a much shorter sequence.
- Branch. Each latent is turned back into tokens by a small local autoregressive branch, and all the branches run in parallel.
- Generate in groups. Strong compression makes generating all latents jointly hard, so latents are produced group by group, each group conditioned on the earlier ones.
Results
Measured end to end on the same GPU, BLD reduces generation FLOPs by more than 80× and raises throughput by more than 6× compared with the similarly sized ELF-L diffusion baseline. Against an autoregressive baseline it delivers more than 6× the throughput and more than 4× lower latency. Local fluency and diversity stay competitive. Long-range coherence remains an open challenge.
Reference
@misc{feng2026denoising,
title={Denoising Blocks, Not Tokens: Efficient Compressed Continuous Diffusion with Branching Token Realization},
author={Xinsong Feng and Peng Du and Zhizhuo Yang and Daniel M. Bikel and Jiayun Wang and Haipeng Chen},
year={2026},
eprint={2610.09311},
archivePrefix={arXiv},
primaryClass={cs.LG}
}