A Benchmark and Framework for Evaluating Next Action Predictions in Spreadsheets

The article introduces a benchmark for evaluating next action predictions in spreadsheets, addressing the lack of predictive code completion in this domain. It presents a curated dataset of 52 sequences with 12,000 actions and outlines an online evaluation method that iteratively predicts user actions based on real-time input. This work is significant for practitioners as it provides a structured framework to enhance predictive capabilities in spreadsheet applications, leveraging various baseline models including zero-shot and fine-tuned language models, while also analyzing the impact of user behavior and context on prediction accuracy.