PIVOTSBench: Evaluating Fine-Grained Interpersonal Relationship Reasoning in Multimodal Large Language Models

PIVOTSBench is a newly introduced benchmark designed to evaluate the fine-grained interpersonal relationship reasoning capabilities of multimodal large language models (MLLMs), utilizing data from Social-IQ 2.0 and YouTube. The benchmark includes auxiliary tasks that assess models' ability to identify visual cues critical for predicting interpersonal dimensions, and it features evaluations on both proprietary and open-source MLLMs through ablation studies focusing on visual modalities and social role information. This development is significant for practitioners as it addresses a gap in understanding how MLLMs can leverage multimodal inputs for nuanced social reasoning, potentially enhancing applications in social AI.