Ensemble Manifold Regularized Multi-Modal Graph Convolutional Network for Cognitive Ability Prediction

Objective: Multi-modal functional magnetic resonance imaging (fMRI) can be used to make predictions about individual behavioral and cognitive traits based on brain connectivity networks. Methods: To take advantage of complementary information from multi-modal fMRI, we propose an interpretable multi-...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:IEEE transactions on biomedical engineering Jg. 68; H. 12; S. 3564 - 3573
Hauptverfasser: Qu, Gang, Xiao, Li, Hu, Wenxing, Wang, Junqi, Zhang, Kun, Calhoun, Vince, Wang, Yu-Ping
Format: Journal Article
Sprache:Englisch
Veröffentlicht: United States IEEE 01.12.2021
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Schlagworte:
Achievement tests
Artificial neural networks
Biological system modeling
Biomarkers
Brain
Brain mapping
Brain modeling
Cognition
Cognition & reasoning
Cognitive ability
Data integration
Deep learning
Embedding
fMRI
functional connectivity
Functional magnetic resonance imaging
graph convolutional networks
interpretable deep learning
Magnetic resonance imaging
Manifolds
Modal data
multi-modal deep learning
Neural networks
Neuroimaging
Performance prediction
Predictions
Predictive models
Regularization
Time series analysis
ISSN:0018-9294, 1558-2531, 1558-2531
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Objective: Multi-modal functional magnetic resonance imaging (fMRI) can be used to make predictions about individual behavioral and cognitive traits based on brain connectivity networks. Methods: To take advantage of complementary information from multi-modal fMRI, we propose an interpretable multi-modal graph convolutional network (MGCN) model, incorporating both fMRI time series and functional connectivity (FC) between each pair of brain regions. Specifically, our model learns a graph embedding from individual brain networks derived from multi-modal data. A manifold-based regularization term is enforced to consider the relationships of subjects both within and between modalities. Furthermore, we propose the gradient-weighted regression activation mapping (Grad-RAM) and the edge mask learning to interpret the model, which is then used to identify significant cognition-related biomarkers. Results: We validate our MGCN model on the Philadelphia Neurodevelopmental Cohort to predict individual wide range achievement test (WRAT) score. Our model obtains superior predictive performance over GCN with a single modality and other competing approaches. The identified biomarkers are cross-validated from different approaches. Conclusion and Significance: This paper develops a new interpretable graph deep learning framework for cognition prediction, with the potential to overcome the limitations of several current data-fusion models. The results demonstrate the power of MGCN in analyzing multi-modal fMRI and discovering significant biomarkers for human brain studies.
Bibliographie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
content type line 23
ISSN:0018-9294
1558-2531
1558-2531
DOI:10.1109/TBME.2021.3077875