A Continuous-Time Markov Chain Framework for Insertion Language Models

The paper introduces a continuous-time Markov chain framework for Insertion Language Models (ILMs), deriving a diffusion-style denoising objective from first principles. This framework allows previous ILM formulations to be viewed as special cases and demonstrates competitive performance against left-to-right generation and masked diffusion models on a synthetic planning task, while enhancing flexibility in sampling. This advancement is significant for practitioners as it provides a more principled approach to ILM design, potentially improving model efficiency and output quality in various applications.