Attention Expansion: Enhancing Keyphrase Extraction from Long Documents with Attention-Augmented Contextualized Embeddings

The paper presents an attention expansion mechanism that enhances keyphrase extraction (KPE) from long documents by augmenting pre-trained language model (PLM) token representations with information from surrounding out-of-context chunks using pre-trained word embeddings. Evaluated across five PLM backbones and multiple benchmark corpora, the approach demonstrates significant improvements in F1 score over state-of-the-art models, indicating its effectiveness in addressing the limitations of traditional PLMs and long-context LLMs for efficient KPE. This mechanism offers a practical solution for practitioners needing to extract salient information from lengthy documents without incurring the high computational costs associated with full-document attention.