Multi-masks and Bi-spaces Reconstruction based Single-Layer Auto-encoder for Heterogeneous Graph Representation Learning

Generative self-supervised learning (SSL) for heterogeneous graph representation has drawn increasing attention due to its powerful capability of capturing rich structural and semantic information without relying on labeled information. However, the multi-layer network structure with multiple attent...

Full description

Saved in:
Bibliographic Details
Published in:Proceedings of ... International Joint Conference on Neural Networks pp. 1 - 9
Main Authors: Zhang, Pei, Zhou, Lihua, Wang, Lizhen, Chen, Hongmei, Xiao, Qing
Format: Conference Proceeding
Language:English
Published: IEEE 30.06.2024
Subjects:
Attention mechanisms
heterogeneous graph representation learning
masked auto-encoder
Neural networks
Representation learning
Self-supervised learning
Semantics
ISSN:2161-4407
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Generative self-supervised learning (SSL) for heterogeneous graph representation has drawn increasing attention due to its powerful capability of capturing rich structural and semantic information without relying on labeled information. However, the multi-layer network structure with multiple attention mechanisms and masking strategies adopted by existing generative SSL methods based on Auto-Encoder, brings semantic confusion and creates dependence of the reconstruction of masked attributes on attribute distinguishability issues. To address these issues, we propose Multi-masks and Bi-spaces reconstruction based Single-Layer Heterogeneous graph AutoEncoder(MBS-HAE), which employs a single-layer masked autoencoder with only semantic attention to learn node embeddings and uses re-masking by randomly selecting nodes to reduce the model's reliance on the distinguishability of input attributes. Meanwhile, MBS-HAE performs bi-spaces reconstruction in both attribute and latent space to improve the certainty of inferring masked data. The extensive experiments on multiple datasets demonstrate the effectiveness and efficiency of the proposed MBS-HAE.
ISSN:2161-4407
DOI:10.1109/IJCNN60899.2024.10650586