CO₂ Emissions and Models Performance: Insights from the Open LLM Leaderboard

The article presents an analysis of CO₂ emissions associated with training various large language models (LLMs) featured on the Open LLM Leaderboard. It highlights that models with larger parameter counts, such as those exceeding 1 billion parameters, tend to have significantly higher emissions, with some training runs estimated to produce over 100 tons of CO₂. This study emphasizes the importance of considering environmental impact alongside performance metrics, encouraging practitioners to adopt more energy-efficient training practices and model architectures to mitigate carbon footprints while optimizing LLM performance.