Finding the Evidence: Discovering Decision-Supporting Tokens for On-Policy Reasoning Distillation

The paper introduces DEAR (Decision-Evidence Aware Reasoning Distillation), a novel approach for on-policy distillation that enhances reasoning transfer by distinguishing between decision-making tokens and evidence tokens. By utilizing student entropy to identify decisions and cosine similarity for evidence discovery, DEAR significantly improves performance on math and code generation benchmarks, achieving up to +2.5 percentage points on math and +5.7 percentage points on code compared to standard on-policy distillation methods. This advancement is crucial for practitioners as it enables more effective knowledge transfer in LLMs, improving their reasoning capabilities in complex tasks.