From Texts to Scores: Tracing the Emergence of Essay Quality Representations in Large Language Models

This study analyzes the internal representations of eight Large Language Models (LLMs) in the context of Automated Essay Scoring (AES) using datasets like ASAP++ and ENEM. It employs techniques such as linear probing and neuron-level analyses to demonstrate that essay quality information is linearly accessible within LLMs, with specific "essay scoring neurons" identified that correlate with essay scores. These insights enhance the interpretability of LLMs in AES, indicating that practitioners can leverage structured representations for improved scoring accuracy and understanding of model behavior.