Multi-masks and Bi-spaces Reconstruction based Single-Layer Auto-encoder for Heterogeneous Graph Representation Learning

Generative self-supervised learning (SSL) for heterogeneous graph representation has drawn increasing attention due to its powerful capability of capturing rich structural and semantic information without relying on labeled information. However, the multi-layer network structure with multiple attent...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Proceedings of ... International Joint Conference on Neural Networks s. 1 - 9
Hlavní autoři: Zhang, Pei, Zhou, Lihua, Wang, Lizhen, Chen, Hongmei, Xiao, Qing
Médium: Konferenční příspěvek
Jazyk:angličtina
Vydáno: IEEE 30.06.2024
Témata:
Attention mechanisms
heterogeneous graph representation learning
masked auto-encoder
Neural networks
Representation learning
Self-supervised learning
Semantics
ISSN:2161-4407
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Generative self-supervised learning (SSL) for heterogeneous graph representation has drawn increasing attention due to its powerful capability of capturing rich structural and semantic information without relying on labeled information. However, the multi-layer network structure with multiple attention mechanisms and masking strategies adopted by existing generative SSL methods based on Auto-Encoder, brings semantic confusion and creates dependence of the reconstruction of masked attributes on attribute distinguishability issues. To address these issues, we propose Multi-masks and Bi-spaces reconstruction based Single-Layer Heterogeneous graph AutoEncoder(MBS-HAE), which employs a single-layer masked autoencoder with only semantic attention to learn node embeddings and uses re-masking by randomly selecting nodes to reduce the model's reliance on the distinguishability of input attributes. Meanwhile, MBS-HAE performs bi-spaces reconstruction in both attribute and latent space to improve the certainty of inferring masked data. The extensive experiments on multiple datasets demonstrate the effectiveness and efficiency of the proposed MBS-HAE.
ISSN:2161-4407
DOI:10.1109/IJCNN60899.2024.10650586