LLM Features Can Hurt GNNs: Concatenation Interference on Homophilous Graph Benchmarks

The study presents findings that integrating LLM-generated node features into GNNs via concatenation can degrade performance on homophilous benchmarks, contrary to common assumptions. Specifically, using SBERT-encoded GPT-4o-mini TAPE features led to a significant accuracy drop of -17.0% on PubMed and -4.3% on Cora with an MLP backbone. The research introduces a measure, Delta_sig, to predict when concatenation is beneficial or harmful, demonstrating that performance degradation is influenced more by LLM discriminability than by homophily, which is critical for practitioners integrating LLMs with GNNs effectively.