Towards Direct Latent-Space Synthesis for Parallel Branches in LLM-Agent Workflows

The paper introduces Parallel-Synthesis, a new framework designed to enhance large language model (LLM) workflows by enabling direct synthesis from key-value (KV) caches produced by parallel worker agents, rather than through traditional text concatenation. This approach includes a cache mapper and a fine-tuned synthesizer adapter, which collectively allow for efficient aggregation of independently generated outputs. The framework demonstrates significant performance improvements, matching or surpassing text-based synthesis on seven out of nine evaluated datasets and reducing time-to-first-token by 2.5x to 11x, indicating its potential for optimizing agent-based systems in AI applications.