ARCO: Adaptive Rubric with Co-Evolution for Multi-Step LLM-Based Agents

The article introduces ARCO (Adaptive Rubric CO-evolution), a novel framework for improving credit assignment in multi-step LLM-based agents through adaptive rubric-based rewards. ARCO features a shared backbone model with a generation head for producing per-step criteria and a scoring head for predicting step-level rewards, enabling dynamic co-evolution of rubric content and scoring functions without requiring step-level labels. The framework demonstrates significant performance improvements on benchmarks like HotpotQA and MuSiQue, providing practitioners with a more interpretable and effective approach to reinforcement learning in LLMs.