Human-AI Teaming Through the Lens of Calibration

The paper presents a study on human-AI teaming focusing on statistical calibration, analyzing how calibration affects the integration of human and AI model predictions. It distinguishes between two frameworks: one that combines predictions and another that delegates responsibility, revealing that existing combination methods fail to maintain human calibration, while delegation shifts the calibration burden to a rejector meta-model, which must be finely calibrated to determine the superior predictor. This research is critical for practitioners as it highlights calibration challenges in designing effective human-AI collaboration systems, emphasizing the need for robust calibration strategies to ensure reliable decision-making.