MAE-EEG-Transformer: A transformer-based approach combining masked autoencoder and cross-individual data augmentation pre-training for EEG classification

Convolutional neural networks (CNN) may not be ideal for extracting global temporal features from non-stationary Electroencephalogram (EEG) signals. The application of the masking-based method in EEG classification is not well studied, and there is a shortage of commonly accepted models for verifyin...

Ausführliche Beschreibung

Gespeichert in:
Bibliographische Detailangaben
Veröffentlicht in:Biomedical signal processing and control Jg. 94; S. 106131
Hauptverfasser: Cai, Miao, Zeng, Yu
Format: Journal Article
Sprache:Englisch
Veröffentlicht: Elsevier Ltd 01.08.2024
Schlagworte:
EEG
Masked autoencoder
Motor imagery
Pre-training
Transformer
Pre-training
Motor imagery
Transformer
Masked autoencoder
EEG
ISSN:1746-8094
Online-Zugang:Volltext
Tags: Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
Beschreibung
Zusammenfassung:Convolutional neural networks (CNN) may not be ideal for extracting global temporal features from non-stationary Electroencephalogram (EEG) signals. The application of the masking-based method in EEG classification is not well studied, and there is a shortage of commonly accepted models for verifying inter-individual results in motor imagery classification tasks. The MAE-EEG-Transformer, a transformer with masking mechanism, is proposed in this article. It pre-trains by randomly masking signals and forces the model to learn semantic features. The pre-trained encoder module is fine-tuned and moved to the classification task to obtain the category of EEG signals. The effectiveness of features with and without pre-training is compared using t-SNE visualization to demonstrate pre-training’s inter-subject efficacy. The MAE EEG Transformer was extensively evaluated across three prevalent datasets in EEG-based motor imagery, demonstrating performance comparable to state-of-the-art models while requiring only approximately 20% of the computational cost (results in Table 1, 2, 3 and 4). •Extracting global temporal geatures from non-stationary EEG signals.•Random masking and reconstruction forces the model to learn semantic features.•Cross-Subject data augmentation pre-training.•Achieving similar performance to SOTA models with 20% of the computational cost.
ISSN:1746-8094
DOI:10.1016/j.bspc.2024.106131