Uniform Diffusion Models Revisited: Leave-One-Out Denoiser and Absorbing State Reformulation

Researchers revisit uniform discrete diffusion and find a mismatch between the usual plug-in bridge parameterization and the standard denoising objective.

They derive leave-one-out versions of the denoiser, posterior, and score, which improve inference through better samplers and temperature scaling.

They also introduce an absorbing-state reformulation that keeps the same joint distribution while making sampling more like masked diffusion.

The result is better language modeling performance without changing the underlying diffusion marginals.