Iterating Toward Better Search: A Two-Agent Simulation Framework for Evaluating Agentic Search Architectures in E-Commerce

A new modular two-agent simulation framework for evaluating conversational shopping assistants has been introduced, allowing for controlled comparisons of responder designs using a consistent buyer agent across various scenarios. Key findings indicate that rolling-window memory significantly outperforms intent-extraction memory in both speed and quality, and that switching the responder's LLM backbone from Gemini 2.5 to Llama 3.3 (70B parameters) results in a minor performance drop. This framework facilitates rapid iteration and targeted improvements in agentic search architectures, providing valuable insights for practitioners in e-commerce AI development.