Towards Scalable Customization and Deployment of Multi-Agent Systems for Enterprise Applications

The article introduces a unified framework for the customization and deployment of multi-agent systems utilizing large language models (LLMs) in enterprise applications. It details a two-stage approach: Agentic Model Customization, which employs continual pretraining, supervised fine-tuning, and preference optimization for adapting compact models to specific domains, and Inference Optimization, which utilizes speculative decoding and FP8 quantization to enhance serving efficiency with a reported 4.48x speedup in throughput. This framework addresses critical challenges in production deployment, making it easier for practitioners to implement scalable, cost-effective solutions in real-world environments while maintaining performance and robustness.