Quantifying Prior Dominance in RAG Systems

The paper introduces the Normalized Context Utilization (NCU) metric to evaluate Retrieval-Augmented Generation (RAG) systems, addressing limitations in current heuristics that conflate contextual information extraction with memory recall. It analyzes models ranging from 1.5B to 72B parameters, revealing that Small Language Models (SLMs) can outperform larger architectures in strict factual extraction, highlighting diminishing returns in scaling. The study emphasizes the importance of model architecture and alignment, noting significant issues with commercial APIs, including negative transfer and reliance on parametric priors over external evidence, which could inform practitioners about the effectiveness of different model sizes in RAG workflows.