Dissecting model behavior through agent trajectories

The paper introduces the "Simple Strands Agent" (SSA), a customizable harness designed to minimize the "intent-execution" gap between AI models and their operational environments. It reproduces or improves upon the pass@1 performance metrics on key benchmarks like SWE-Pro and Terminal-Bench-2 across various model families (Claude, Gemini, GPT, Grok, Qwen) and analyzes 138,000 agent trajectories to uncover nuanced differences in problem-solving behaviors among models. This work highlights the importance of harness design in aligning model capabilities with practical execution, providing insights that can enhance the performance of AI agents in real-world applications.