Generalization of Fine-Tuned Uncertainty Communication and Metacognition in Large Language Models

This study investigates the impact of supervised fine-tuning on the uncertainty communication capabilities of large language models. Two models were fine-tuned on diverse tasks, showing improved alignment between confidence levels and answer correctness, particularly in single-question confidence estimation and pairwise comparisons. The findings suggest that while fine-tuning enhances metacognitive performance, the transfer of skills between different confidence tasks is limited, indicating the potential benefit of multitask training for broader applicability in AI applications.