Evaluation Cards: An Interpretive Layer for AI Evaluation Reporting

The paper introduces \EvalCards{}, an operational reporting layer designed to unify and standardize AI evaluation reporting by integrating benchmark metadata, evaluation run data, and model metadata into a cohesive framework. Key features include a reporting schema derived from a review of 52 papers and stakeholder interviews, and the implementation of interpretive signals such as reproducibility and score comparability. This framework addresses critical gaps in AI evaluation transparency and comparability, enabling practitioners to better assess model performance across diverse sources and improve the reliability of their evaluations.