Attention Might Offer Little Benefit for Graph Node Classification

Attention mechanisms have achieved remarkable success in language models and have since been widely adopted in vision, speech, and multimodal learning. This trend has extended to graph learning, where attention-based models such as Graph Attention Networks (GAT) and Graph Transformers are now prevalent. textbf{This position paper argues that attention mechanisms may not be as beneficial for graph node classification as commonly believed.} Through systematic ablation studies, we find that attention often provides negligible or even detrimental gains compared to simpler alternatives, with the only notable exception being graphs whose node features are language word embeddings. This suggests that the benefit of attention is largely limited outside language-related applications. We examine attention at three scales: 1-hop (GAT-style), Inception-style, and global mechanisms. We further analyze potential explanations for these results, including the limitations of gradient-based optimization and the fundamental differences between language and graph. Overall, these findings suggest that the prevailing enthusiasm for attention in graph node classification may be overstated, motivating a more critical and evidence-driven re-evaluation of its adoption. The code for all experiments is available at https://github.com/Qin87/ScaleNet/tree/July25.