Entropy-Gradient Inversion: Moving Toward Internal Mechanism of Large Reasoning Models

The paper introduces "Entropy-Gradient Inversion," a concept that reveals a negative correlation between token entropy and logit gradients, serving as a geometric indicator of reasoning capabilities in Large Reasoning Models (LRMs). It proposes a new reinforcement learning technique called "Correlation-Regularized Group Policy Optimization (CorR-PO)," which integrates this inversion into reward regularization, demonstrating improved performance on reasoning benchmarks across various model sizes. This work is significant for practitioners as it offers a novel approach to enhancing reasoning in LLMs, addressing limitations in current reinforcement learning methods.