Turning Intent into Specifications: A Benchmark and an Interactive User-Assistant Agent

The paper introduces SpecBench, a benchmark designed to assess an agent's capability to convert vague user intent into structured, executable specifications through interactive dialogue. It highlights the limitations of current agents in effectively collaborating with users and presents Buddy, a user-assistant agent that employs classical morphological analysis to systematically decompose user intent and facilitate decision-making. This approach underscores the importance of enhancing agent-user collaboration in the development of future AI systems, moving beyond mere code generation to a more nuanced understanding of user preferences.