MINCE: Shrinking LLM Evaluation Datasets via Few-Model Monte Carlo Calibration

MINCE (Monte Carlo Informed N-sizing for Compact Evaluation) has been introduced as a novel method to optimize evaluation datasets for large language models (LLMs) by reducing the required calibration model size without needing a prediction layer. The approach utilizes Monte Carlo simulations to identify a minimal subset of evaluation items, achieving significant reductions in benchmark sizes—54% for IFEVAL, 89% for MMLU, and 70% for GSM8K—while maintaining accuracy drift within acceptable limits. This results in substantial evaluation speedups (2.7–8.1× on GPUs and 1.7–2.0× on NPUs), making it particularly valuable for practitioners seeking efficient evaluation strategies for various model configurations.