BlockGen: Flexible Blockwise Sequence Modeling with Hybrid Samplers

BlockGen is a blockwise discrete diffusion model trained on mixed block sizes, comparing masked and uniform diffusion for sequence generation.

The authors also introduce AR-informed predictor-corrector sampling, which combines autoregressive and diffusion predictions to regenerate tokens that are likely wrong.

Experiments show uniform diffusion has an edge under ancestral sampling, especially with few steps, in blockwise generation.

With AR-informed sampling at higher compute, masked diffusion can match or slightly outperform uniform diffusion on some benchmarks.

The work suggests discrete diffusion can be improved without a separate verifier, and tests whether uniform-state diffusion still matters in blockwise settings.