Post-Training Recipe, More Than Model Family, Shapes Multi-Agent LLM Conversational Behavior

This study reveals that the post-training recipe significantly influences conversational behavior in multi-LLM systems, challenging the notion that model family alone ensures behavioral diversity. Using a dataset of 940,000 chains across 11 checkpoints and a factorial of 1.6 million chains from the same base Llama model, the research demonstrates that conversational responses can vary by up to 18% based on the specific partner model, indicating that practitioners should consider post-training strategies when selecting models for multi-agent interactions. This finding underscores the importance of evaluating LLMs beyond family classifications to enhance the effectiveness of multi-agent systems.