GAGPO: Generalized Advantage Grouped Policy Optimization

GAGPO (Generalized Advantage Grouped Policy Optimization) is a novel critic-free reinforcement learning method designed to enhance credit assignment in multi-turn environments by enabling precise, step-aligned temporal credit assignment. It utilizes a non-parametric grouped value proxy to compute TD/GAE-style temporal advantages, facilitating backward propagation of outcome supervision without auxiliary value models. Experimental results on ALFWorld and WebShop indicate that GAGPO significantly outperforms existing reinforcement learning baselines, demonstrating faster learning and improved optimization dynamics, which is crucial for practitioners aiming to develop more efficient multi-turn agents.