ChartFI: Benchmarking Faithfulness and Insightfulness of Chart Descriptions from Multimodal Large Language Models

The article introduces the Chart Faithfulness and Insightfulness Benchmark (ChartFI-Bench), which addresses the limitations of current benchmarks in evaluating chart descriptions generated by multimodal large language models (MLLMs). It presents a dataset of 896 chart-description pairs emphasizing dimensions such as factual accuracy and chart-text complementarity, and proposes four evaluation metrics: Faithfulness, Coverage, Informativeness, and Acuity. This benchmark is crucial for practitioners as it provides a more rigorous framework for assessing the quality of automated chart descriptions, ultimately enhancing the reliability and utility of MLLMs in data visualization contexts.