The Origins of Stochasticity: Comprehensive Investigations on Uncertainty Quantification for Large Language Models

The paper presents a new granular uncertainty taxonomy for Large Language Models (LLMs), categorizing uncertainty into input-level, parameter-level, token-level, and decoding-process sources. It evaluates 21 uncertainty quantification methods across three LLM families—Qwen3, Llama 3.2, and DeepSeek-V3—using benchmarks like TriviaQA and GSM8K, finding that consensus-based methods outperform others and that larger models tend to have lower uncertainty estimates. This work is significant for practitioners as it provides a structured framework for assessing and mitigating uncertainty in LLM applications, enhancing predictive credibility in deployment.