Calibration Is Not Control: Why LLM-Agent Oversight Needs Intervention

The article presents a new approach to oversight for LLM agents, arguing that traditional scalar risk prediction methods are inadequate for effective intervention. It introduces the concept of intervention advantage and formalizes it through prefix branching, demonstrating that action-conditioned control can significantly reduce control regret, achieving a reduction from 0.506 to 0.110 on the ALFWorld benchmark. This shift towards action-conditioned value estimation emphasizes the need for more nuanced oversight mechanisms in LLM deployment, enhancing the reliability and effectiveness of interventions.