SAGE: Stochastic Prompt Optimization via Agent-Guided Exploration

The paper introduces SAGE (Stochastic Prompt Optimization via Agent-Guided Exploration), a framework for optimizing prompts through stochastic search strategies, including error-informed random search and a genetic algorithm. SAGE employs a multi-agent system that integrates diagnostic code execution to enhance performance across various benchmarks, notably improving retention rates in a mental-health chatbot through continuous optimization. This approach highlights the importance of combining qualitative diagnostics with quantitative validation for effective open-ended dialogue systems, offering practitioners a novel method for refining AI interaction without modifying underlying model parameters.