Reinforcement-aware Knowledge Distillation for LLM Reasoning

The paper introduces Reinforcement-aware Knowledge Distillation (RLAD) for large language models, which enhances the distillation process by addressing issues of distribution mismatch and objective interference when integrating reinforcement learning (RL) with knowledge distillation. The method employs Trust Region Ratio Distillation (TRRD), which utilizes a PPO/GRPO-style likelihood-ratio objective to guide student models selectively, achieving superior performance on logic reasoning and math benchmarks compared to traditional methods. This approach is significant for practitioners as it offers a more efficient way to distill powerful RL-optimized models into smaller, more deployable versions without sacrificing reasoning capabilities.