Efficient Message Passing Algorithm and Architecture Co-Design for Graph Neural Networks

Graph neural networks (GNNs) are a promising method for learning graph representations and demonstrate remarkable performance on various graph-related tasks. Existing typical GNNs exploit the neighborhood message passing scheme that subtly aggregates feature messages from neighbor nodes to update th...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE transactions on emerging topics in computational intelligence Ročník 9; číslo 1; s. 889 - 903
Hlavní autoři: Zou, Xiaofeng, Chen, Cen, Zhang, Luochuan, Li, Shengyang, Zhou, Joey Tianyi, Wei, Wei, Li, Kenli
Médium: Journal Article
Jazyk:angličtina
Vydáno: Piscataway IEEE 01.02.2025
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Accuracy
Algorithm and architecture co-design
Algorithm design and theory
Algorithms
Co-design
Computational modeling
Computer architecture
dynamic pruning
Energy consumption
Graph neural networks
Graph representations
Graphical representations
Hardware
Machine learning
Message passing
Neural networks
Optimization techniques
Performance enhancement
Power demand
Task complexity
ISSN:2471-285X, 2471-285X
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Graph neural networks (GNNs) are a promising method for learning graph representations and demonstrate remarkable performance on various graph-related tasks. Existing typical GNNs exploit the neighborhood message passing scheme that subtly aggregates feature messages from neighbor nodes to update the node representations. Despite the effectiveness of this scheme, its complex computational model heavily relies on the graph structure, which hinders their scaling to realistic large-scale graph applications. Although several custom accelerators have been proposed to speed up GNNs, these hardware-specific optimization techniques fail to address the fundamental problem of high computational complexity in GNNs. To tackle this challenge, we propose a dedicated algorithm-architecture co-design framework, dubbed MePa, which aims to improve GNN execution efficiency by coordinating algorithm- and hardware-level innovations. Specifically, with an in-depth analysis of GNN message-passing algorithms and potential speedup opportunities, we first propose an efficient message-passing algorithm that can dynamically prune task-irrelevant graph data at multiple granularity, including channel/edge/node-wise. This approach significantly reduces the overall complexity of GNN with negligible accuracy loss. A novel architecture is designed to support dynamic pruning and translate it into performance improvements. Elaborate pipelines and specialized optimizations jointly improve performance and decrease energy consumption. Compared to the state-of-the-art GNN accelerator AWB-GCN, MePa achieves on average <inline-formula><tex-math notation="LaTeX">\text{1.95} \times</tex-math></inline-formula> speedups and <inline-formula><tex-math notation="LaTeX">\text{2.6} \times</tex-math></inline-formula> energy efficiency.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2471-285X
2471-285X
DOI:10.1109/TETCI.2024.3420692