Evaluation of Small Language Models for Arabic Language Processing

The paper evaluates twelve Small Language Models (SLMs) for Arabic natural language processing, introducing a benchmark of 240 test items across various domains and skills. Gemma 3 (12B) achieved the highest score of 4.548/5, with findings indicating that stronger Arabic alignment and instruction-following capabilities correlate with better performance, regardless of model size. This benchmark serves as a structured reference for developing efficient and culturally relevant Arabic AI systems, highlighting common failure patterns in lower-performing models.