Getting Better at Working With You: Compiling User Corrections into Runtime Enforcement for Coding Agents

The article introduces TRACE (Test-time Rule Acquisition and Compiled Enforcement), a novel skill-layer pipeline for coding agents that compiles user corrections into runtime checks to enhance compliance with user preferences. Evaluated on ClawArena and MemoryArena tasks, TRACE significantly reduces preference violations from 100% to as low as 2%, outperforming existing memory-based methods. This advancement is crucial for practitioners as it minimizes the need for users to repeatedly provide corrections, thereby improving the usability and reliability of interactive LLM agents in coding environments.