VoidPadding: Let [VOID] Handle Padding in Masked Diffusion Language Models so that [EOS] Can Focus on Semantic Termination

VoidPadding introduces a novel padding token, [VOID], to separate the roles of padding and semantic termination in Masked Diffusion Language Models (MDLMs), specifically addressing the issue of [EOS] token overflow during large-block decoding. Implemented on the Dream-7B-Instruct model, VoidPadding achieves a significant improvement of +17.84 points in a four-task mean across mathematical reasoning and code generation benchmarks, while also reducing decoding NFE by 55.7% compared to previous methods. This approach enhances the efficiency and effectiveness of instruction tuning in MDLMs, making it a valuable development for practitioners in AI model training.