Researchers Rethink Masking in Diffusion-Based Text Generation

Diffusion language models waste useful probability information by forcing tokens into binary masked or fixed states. This paper shows how letting tokens evolve through soft distributions—supervised at every refinement step—improves accuracy, revisability, and efficiency across tasks.