Self-Evolution for Multi-Turn Tool-Calling Agents via Divergence-Point Preference Learning

The paper presents ToolGraph, a framework for enhancing multi-turn tool-using agents through self-improvement via divergence-point preference learning (DPO). ToolGraph integrates schema-derived topology and transition weights from successful rollouts to improve tool selection and preference updates, achieving a weighted average reward increase from 0.304 to 0.338 across 375 tau2-bench tasks, with ToolGraph+DPO further improving to 0.355. This work is significant for AI practitioners as it addresses the challenges of orchestration and preference learning in tool-using agents, providing a structured approach to enhance performance in complex tasks.