HAKARI-Bench: A Lightweight Benchmark for Comparing Retrieval Architectures and Efficiency Settings under Unified Conditions

HAKARI-Bench is a newly released lightweight benchmark designed for evaluating retrieval architectures and efficiency settings under unified conditions, featuring 35 benchmarks and 551 tasks across 43 languages. It allows for model-agnostic comparisons of five retrieval families (BM25, dense, sparse, late interaction, rerankers) and their efficiency variants, achieving a high correlation with established benchmarks (Spearman >0.97) across 55 models. This tool is significant for practitioners as it facilitates rapid model selection and regression detection without the overhead of comprehensive evaluations, thus aiding in the optimization of retrieval-augmented generation and semantic search systems.