Know Thy Reasoner: Not All Language Models Explore Alike

The paper introduces a framework for optimizing reasoning strategies in language models (LLMs) based on their diversity profiles, which describe the distribution of probability mass across solution approaches. It demonstrates that low-diversity models benefit from depth-based refinement, while high-diversity models require breadth or stronger signals for effective exploration. This insight is crucial for practitioners, as it informs the design of exploration strategies tailored to the characteristics of specific LLMs, potentially enhancing their reasoning capabilities.