Fix the Structural Bottleneck: Context Compression via Explicit Information Transmission

The paper introduces ComprExIT, a novel context compression framework designed to enhance the efficiency of long-context LLMs by addressing structural bottlenecks in existing LLM-based compressors. ComprExIT improves contextual information preservation through adaptive feature selection and a globally coordinated transport plan, yielding up to 18.5% better average F1 scores across 12 datasets while only increasing trainable parameters by approximately 1% and achieving over 2x faster compression than current leading methods. This advancement is significant for practitioners looking to optimize the performance and deployment of LLMs with long context capabilities.