Toward Instructions-as-Code: Understanding the Impact of Instruction Files on Agentic Pull Requests

The paper investigates the impact of instruction files on the performance of AI agents, such as GitHub Copilot, in generating pull requests (Agentic-PRs) by analyzing 15,549 PRs across 148 projects in the AIDev dataset. Results show mixed outcomes: while 27.7% of projects increased their merge rate by at least 20% with structured instruction files, 26.35% experienced a decrease. The findings suggest a need for further research on developing effective instruction files as a structured software engineering activity, termed "Instructions-as-Code," to enhance AI agent performance in collaborative coding tasks.