When Confidence Takes the Wrong Path: Diagnosing Retrieval-State Lock-In in RAG

The paper introduces the concept of "retrieval-state lock-in," a failure mode in retrieval-augmented generation (RAG) systems where confidence is misinterpreted due to stable errors arising from defective retrieval states. The authors provide a diagnostic framework that separates the components of confidence—answer surface, retrieved evidence, and retrieval state—highlighting that 42% of errors in their ontology-guided knowledge-graph RAG (KG-RAG) system show zero answer dispersion despite evidence checks indicating issues. This approach offers a method for practitioners to enhance decision-making in RAG systems by ensuring that all three components align before accepting an answer, achieving a 91.9% pooled precision, though at the cost of coverage, certifying only 7.7% of answers as low-risk.