PLAIground: SLO-Driven Runtime Model Selection for Compound AI Systems in the Edge-Cloud-Space Continuum

PLAIground is a newly introduced framework designed for runtime model selection in Compound AI systems across the edge-cloud-space continuum. It features the Compoundable AI Model (CAIM) abstraction, allowing dynamic model switching without altering workflow semantics, and incorporates the Pixie algorithm, which optimizes model selection based on Service Level Objectives (SLOs). Evaluation results indicate that Pixie achieves up to 91.3% accuracy while ensuring compliance with cost and latency requirements, significantly outperforming fixed-model strategies that can exceed budgets by 21x or fall short on accuracy by 4%.