Regression Language Models for Code

The article presents a Regression Language Model (RLM) that leverages a frozen LLM encoder to predict numeric outcomes from code execution across multiple programming languages, including Python and C++. The model, based on T5Gemma with 300 million parameters, achieves over 0.9 Spearman-rank on competitive programming submissions and an average Spearman-rank of over 0.5 across 24 programming languages from CodeNet. This unified approach demonstrates significant performance in areas traditionally dominated by heavy feature engineering and graph neural networks, making it relevant for practitioners focused on optimizing code performance and hardware architecture predictions.