Energy-Based Transformers as Predictors of Reading Difficulty

The article introduces energy-based transformers as a novel approach for predicting reading difficulty in computational psycholinguistics, establishing a connection to associative memory models like Hopfield networks. The study demonstrates that energy measures from these transformers provide a robust prediction of reading times across various corpora, outperforming traditional metrics such as surprisal and attention entropy, and effectively capturing asymmetries in processing. This advancement may streamline the assessment of cognitive load in language processing, offering practitioners a unified metric for model evaluation.