Early-Token Confidence Predicts Reasoning Quality in Multi-Agent LLM Debate

The paper presents a study on using early-token confidence, derived from token-level log-probabilities, to predict reasoning quality in multi-agent LLM systems during debates. The results indicate that early-token confidence is a superior predictor of reasoning quality compared to full-sequence statistics, particularly in the initial tokens generated. This finding is significant for practitioners as it offers a lightweight method to estimate reasoning reliability, enhancing the evaluation processes in open-ended tasks without reference answers.