UniversalRAG: Retrieval-Augmented Generation over Corpora of Diverse Modalities and Granularities

UniversalRAG is a newly introduced Retrieval-Augmented Generation framework that enables retrieval and integration of knowledge from diverse modalities and granularities, overcoming the limitations of existing methods that typically focus on single modalities. It employs a modality-aware routing mechanism to dynamically select the most relevant corpus for retrieval, addressing modality gaps and enhancing response accuracy. Validation across 10 benchmarks demonstrates its superior performance compared to both modality-specific and unified approaches, making it a significant advancement for practitioners looking to build more robust and versatile AI systems.