ProvenanceGuard: Source-Aware Factuality Verification for MCP-Based LLM Agents

ProvenanceGuard is a new source-aware verification tool designed for Model Context Protocol (MCP)-based LLM agents, addressing the issue of cross-source conflation where claims are attributed to incorrect sources. It analyzes MCP traces to decompose answers into atomic claims, checks their support using natural language inference, and compares stated attributions with actual sources, achieving a block F1 score of 0.802 and source accuracy of 0.858 on medical-domain traces. This tool is significant for practitioners as it enhances factuality verification by incorporating source attribution, providing a more reliable framework for LLM agents that utilize diverse evidence sources.