SELECTOR: Heterogeneous graph network with convolutional masked autoencoder for multimodal robust prediction of cancer survival

Accurately predicting the survival rate of cancer patients is crucial for aiding clinicians in planning appropriate treatment, reducing cancer-related medical expenses, and significantly enhancing patients’ quality of life. Multimodal prediction of cancer patient survival offers a more comprehensive...

Full description

Saved in:
Bibliographic Details
Published in:Computers in biology and medicine Vol. 172; p. 108301
Main Authors: Pan, Liangrui, Peng, Yijun, Li, Yan, Wang, Xiang, Liu, Wenjuan, Xu, Liwen, Liang, Qingchun, Peng, Shaoliang
Format: Journal Article
Language:English
Published: United States Elsevier Ltd 01.04.2024
Elsevier Limited
Subjects:
Cancer
Coders
Convolutional mask encoder
Embedding
Graph theory
Heterogeneous graph
Humans
Internal Medicine
Missing
Modules
Multimodal
Neoplasms - diagnostic imaging
Other
Patients
Predictions
Quality of Life
Reconstruction
Robust
Robustness
Survival
Missing
Heterogeneous graph
Multimodal
Robust
Convolutional mask encoder
ISSN:0010-4825, 1879-0534, 1879-0534
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Accurately predicting the survival rate of cancer patients is crucial for aiding clinicians in planning appropriate treatment, reducing cancer-related medical expenses, and significantly enhancing patients’ quality of life. Multimodal prediction of cancer patient survival offers a more comprehensive and precise approach. However, existing methods still grapple with challenges related to missing multimodal data and information interaction within modalities. This paper introduces SELECTOR, a heterogeneous graph-aware network based on convolutional mask encoders for robust multimodal prediction of cancer patient survival. SELECTOR comprises feature edge reconstruction, convolutional mask encoder, feature cross-fusion, and multimodal survival prediction modules. Initially, we construct a multimodal heterogeneous graph and employ the meta-path method for feature edge reconstruction, ensuring comprehensive incorporation of feature information from graph edges and effective embedding of nodes. To mitigate the impact of missing features within the modality on prediction accuracy, we devised a convolutional masked autoencoder (CMAE) to process the heterogeneous graph post-feature reconstruction. Subsequently, the feature cross-fusion module facilitates communication between modalities, ensuring that output features encompass all features of the modality and relevant information from other modalities. Extensive experiments and analysis on six cancer datasets from TCGA demonstrate that our method significantly outperforms state-of-the-art methods in both modality-missing and intra-modality information-confirmed cases. Our codes are made available at https://github.com/panliangrui/Selector. [Display omitted] •Construct a multimodal heterogeneous graph and propose the idea of a meta-path to perform edge reconstruction of features to fully consider the feature information of the edge of the graph and the effective embedding of nodes.•We propose a convolutional masked autoencoder (CMAE) to process the heterogeneous image after feature reconstruction. It mainly relies on the idea of sparse convolution in feature extraction. Only unmasked features are extracted.•A feature cross-communication module is designed to establish communication between multiple modalities, so that the output features include all features of the modality as well as relevant information of other modalities.•SELECTOR can effectively predict cancer patient survival in the presence of both within-modality and missing-modality data. Through extensive experimental verification, we found that this framework achieved the best prediction accuracy on six cancer multimodal datasets of TCGA.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:0010-4825
1879-0534
1879-0534
DOI:10.1016/j.compbiomed.2024.108301