When Do LLMs Reason? A Dynamical Systems View via Entropy Phase Transitions

The paper presents a novel framework called EDRM (Entropy Dynamics-based Reasoning Manifold) that leverages early-stage entropy dynamics to optimize reasoning strategies in LLMs during generation. By analyzing entropy patterns, EDRM adaptively selects inference methods, achieving 41-55% token reduction and up to 4.7% accuracy improvement across 15 benchmarks and 4 different LLM architectures. This approach emphasizes the need for selective reasoning invocation, suggesting that not all tasks benefit from chain-of-thought reasoning, thus providing a more efficient way to manage LLM inference.