n-HDP-GNN: community-aware Bayesian clustering for over-smoothing-resilient, communication-efficient distributed GNNs
Saved in:
| Title: | n-HDP-GNN: community-aware Bayesian clustering for over-smoothing-resilient, communication-efficient distributed GNNs |
|---|---|
| Authors: | Khalili Sadaghiani, Vahab, Nunez-Yanez, Jose Luis |
| Source: | Journal of Supercomputing. 81(16) |
| Subject Terms: | Graph neural network (GNN), Nested Hierarchical Dirichlet Process (n-HDP), Over-smoothing, Community detection, Distributed training, High-performance computing (HPC) |
| Description: | Deep graph neural networks (GNNs) often suffer from over-smoothing, where node embeddings homogenize as depth grows. We present n-HDP-GNN, a probabilistic, community-aware architecture that couples Louvain coarsening with a nested Hierarchical Dirichlet process to learn soft responsibilities that gate message passing. Multi-level attention (node/community/global) then aggregates features while preserving separability. This selective diffusion delays over-smoothing-quantified once using MADGap-and, in distributed training, reduces cross-partition communication by lowering the cross-partition edge ratio and increasing edge-reduction. We evaluate across seven benchmarks spanning citation networks, a co-purchase network, and three large-scale tasks, under supervised, semi-supervised, and label-scarce regimes, against strong baselines. The proposed model delivers over 5% gain in accuracy, greater robustness, and a superior ability to capture long-range dependencies and subtle patterns. Deployed on a multi-node CPU cluster with PyTorch DDP, n-HDP-GNN attains + 11% higher throughput than the best competitor at matched accuracy, demonstrating that the same community-aware gating curbs over-smoothing and improves communication efficiency on commodity interconnects. Together, these results show that probabilistic, community-aware gating yields depth-robust representations without sacrificing scalability: mid-depth performance is strengthened, deep-depth degradation is reduced, and systems metrics improve in tandem turning a representation-level idea into a practical approach for training deep GNNs on modest clusters. |
| File Description: | |
| Access URL: | https://urn.kb.se/resolve?urn=urn:nbn:se:liu:diva-219780 https://doi.org/10.1007/s11227-025-08017-9 |
| Database: | SwePub |
Full Text Finder 
Nájsť tento článok vo Web of Science | Abstract: | Deep graph neural networks (GNNs) often suffer from over-smoothing, where node embeddings homogenize as depth grows. We present n-HDP-GNN, a probabilistic, community-aware architecture that couples Louvain coarsening with a nested Hierarchical Dirichlet process to learn soft responsibilities that gate message passing. Multi-level attention (node/community/global) then aggregates features while preserving separability. This selective diffusion delays over-smoothing-quantified once using MADGap-and, in distributed training, reduces cross-partition communication by lowering the cross-partition edge ratio and increasing edge-reduction. We evaluate across seven benchmarks spanning citation networks, a co-purchase network, and three large-scale tasks, under supervised, semi-supervised, and label-scarce regimes, against strong baselines. The proposed model delivers over 5% gain in accuracy, greater robustness, and a superior ability to capture long-range dependencies and subtle patterns. Deployed on a multi-node CPU cluster with PyTorch DDP, n-HDP-GNN attains + 11% higher throughput than the best competitor at matched accuracy, demonstrating that the same community-aware gating curbs over-smoothing and improves communication efficiency on commodity interconnects. Together, these results show that probabilistic, community-aware gating yields depth-robust representations without sacrificing scalability: mid-depth performance is strengthened, deep-depth degradation is reduced, and systems metrics improve in tandem turning a representation-level idea into a practical approach for training deep GNNs on modest clusters. |
|---|---|
| ISSN: | 09208542 15730484 |
| DOI: | 10.1007/s11227-025-08017-9 |