ERAlign: Energy-based Representation Alignment of GNNs and LLMs on Text-attributed Graphs

The paper introduces ERAlign, an Energy-based Representation Alignment framework designed to enhance the integration of Graph Neural Networks (GNNs) and Large Language Models (LLMs) on Text-attributed Graphs (TAGs). By projecting GNN-encoded structures and LLM-derived text embeddings into a shared latent space and optimizing alignment through an Energy-based Model objective, ERAlign achieves superior representation consistency, demonstrated through state-of-the-art performance across eight TAG datasets with varying supervision levels. This approach addresses representation drift and improves generalization, making it a significant advancement for practitioners working on multi-modal learning tasks involving graphs and textual data.