Coding
RigorBench: Benchmarking Engineering Process Discipline in Autonomous AI Coding Agents
RigorBench is introduced as the first benchmark specifically designed to assess process discipline in autonomous AI coding agents, focusing on five key pillars: Planning Fidelity, Verification Coverage, Recovery Efficiency, Abstention Quality, and Atomic Transition Integrity. The benchmark comprises 30 tasks across various categories and demonstrates that structured process discipline enhances process quality scores by an average of 41% and improves downstream outcome correctness by 17%. This release offers critical insights for practitioners, emphasizing the importance of engineering discipline in the development of reliable AI coding agents, alongside traditional outcome correctness metrics.
benchmarkagentscoding