Where You Inject Diversity Matters: A Unified Framework for Diverse Generation

The paper presents a unified framework for enhancing diversity in open-ended generation tasks by characterizing test-time diverse generation methods based on the source of diversity introduced during generation. The authors propose a novel approach that generates diverse intermediate specifications before conditioning on them to produce final outputs, demonstrating improved diversity across five tasks and four backbone models while maintaining output quality. This framework emphasizes the importance of both the diversity of sources and their effective transmission to the final output, providing practitioners with insights to develop more diverse generation systems in large language models.