Where to Place the Query? Unveiling and Mitigating Positional Bias in In-Context Learning for Diffusion LLMs via Decoding Dynamics

This paper investigates the effects of query placement in In-Context Learning (ICL) for Diffusion Large Language Models (dLLMs), highlighting that query position acts as a first-order variable affecting generation quality. It introduces Average Confidence ($\overline{C}$) as a novel metric for evaluating decoding processes and presents Auto-ICL, an adaptive routing strategy for optimizing query placement without the need for ground-truth labels. These advancements are significant for practitioners as they enhance the understanding of dLLM behavior and provide methods to improve model performance in various reasoning and perception tasks.