$x$-Prediction Flow: Efficient Continuous Decoding for Masked Diffusion Language Models
Weitian Wang, Lianlei Shan, Shubham Rai, Cecilia De La Parra, Akash Kumar
dlm-study
Research metadataShow detailsHide details
- Affiliations
- Not available
- Published
- 2026-06-27
- Processed
- 8/30/2026, 11:59:09 AM
- Analysis model
- Not available
- Analysis status
- partial
- Local PDF artifact
- papers/pdf/2026/x-prediction-flow-efficient-continuous-decoding-for-masked-diffusion-language-models.pdf
Abstract
Masked diffusion language models (MDLMs) generate text by iteratively unmasking tokens, but their standard decoder reduces each step to a binary action: a position is either committed to a single token or left fully masked, discarding rich predictive information rather than carrying it forward, and forcing premature, irrevocable commitments that lead to poor performance under a limited decoding budget. In this paper, we reinterpret mask prediction as a clean-state prediction ($x$-prediction) and show that it can be used to induce a continuous flow in the input embedding space. Building on this view, we propose a continuous decoding framework for MDLMs where tokens can accumulate partial progress at each diffusion step and remain revisable. To match the uneven contextual constraints across positions in language, we replace the globally synchronous schedule in image diffusion with a confidence-based asynchronous update in which the diffusion progress is token-wise accumulated. Additionally, we introduce a lightweight policy network and formulate its training as a reinforcement learning problem. Applied to pretrained LLaDA, our decoder retains 83--97% of full-budget accuracy using under 15% of the diffusion steps, largely outperforming discrete mask-prediction decoding at matched budgets.
Summary
이 논문은 Masked Diffusion Language Model(MDLM)이 “마스크를 토큰으로 확정할지 말지”라는 이진 선택만 하는 것이 비효율적이라고 보고, 각 위치의 상태를 [MASK] 임베딩에서 예측된 clean token 임베딩 쪽으로 조금씩 이동시키는 연속적이고 수정 가능한 디코딩 방식을 제안합니다.