INFRAMIND: Infrastructure-Aware Multi-Agent Orchestration

INFRAMIND is a newly proposed framework for multi-agent orchestration that incorporates real-time infrastructure awareness to optimize model selection and scheduling based on dynamic serving conditions. It employs a hierarchical constrained Markov Decision Process (MDP) solved via reinforcement learning, achieving up to 7.6 percentage points improvement in accuracy at low load and maintaining 99.9% Service Level Objective (SLO) compliance under high load, significantly reducing latency by up to 7 times compared to previous methods. This approach addresses resource underutilization in shared GPU clusters by adapting to runtime signals such as queue depths and latencies, making it crucial for practitioners aiming to enhance efficiency in AI model deployment.