Steer, Don't Solve: Training Small Critic Models for Large Code Agents

The paper introduces a novel approach to enhancing large code agents by incorporating a small critic model that provides intra-trajectory feedback through Supervised Fine-Tuning, rather than relying on post-hoc evaluations. The critic, trained on CWM-32B trajectories, demonstrates significant performance improvements on SWE-bench Verified, achieving gains of up to +5.2 points on Qwen agents while reducing training costs by 30-92 times compared to traditional methods. This approach highlights the potential for smaller, specialized models to optimize training efficiency and accuracy in large-scale code generation tasks.