Token-Level LLM Collaboration via FusionRoute

FusionRoute is a token-level multi-LLM collaboration framework that employs a lightweight router to dynamically select the most suitable expert model at each decoding step while also refining the output through logit addition from a complementary generator. This approach addresses the limitations of fixed expert outputs by expanding the effective policy class, leading to improved performance across diverse benchmarks, including mathematical reasoning and code generation, when tested on the Llama-3 and Gemma-2 models. For practitioners, FusionRoute offers a more efficient method for leveraging multiple LLMs, potentially reducing the need for large-scale general-purpose models while maintaining competitive performance in specialized tasks.