MAE-EEG-Transformer: A transformer-based approach combining masked autoencoder and cross-individual data augmentation pre-training for EEG classification

Convolutional neural networks (CNN) may not be ideal for extracting global temporal features from non-stationary Electroencephalogram (EEG) signals. The application of the masking-based method in EEG classification is not well studied, and there is a shortage of commonly accepted models for verifyin...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:Biomedical signal processing and control Ročník 94; s. 106131
Hlavní autoři: Cai, Miao, Zeng, Yu
Médium: Journal Article
Jazyk:angličtina
Vydáno: Elsevier Ltd 01.08.2024
Témata:
EEG
Masked autoencoder
Motor imagery
Pre-training
Transformer
Pre-training
Motor imagery
Transformer
Masked autoencoder
EEG
ISSN:1746-8094
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Convolutional neural networks (CNN) may not be ideal for extracting global temporal features from non-stationary Electroencephalogram (EEG) signals. The application of the masking-based method in EEG classification is not well studied, and there is a shortage of commonly accepted models for verifying inter-individual results in motor imagery classification tasks. The MAE-EEG-Transformer, a transformer with masking mechanism, is proposed in this article. It pre-trains by randomly masking signals and forces the model to learn semantic features. The pre-trained encoder module is fine-tuned and moved to the classification task to obtain the category of EEG signals. The effectiveness of features with and without pre-training is compared using t-SNE visualization to demonstrate pre-training’s inter-subject efficacy. The MAE EEG Transformer was extensively evaluated across three prevalent datasets in EEG-based motor imagery, demonstrating performance comparable to state-of-the-art models while requiring only approximately 20% of the computational cost (results in Table 1, 2, 3 and 4). •Extracting global temporal geatures from non-stationary EEG signals.•Random masking and reconstruction forces the model to learn semantic features.•Cross-Subject data augmentation pre-training.•Achieving similar performance to SOTA models with 20% of the computational cost.
ISSN:1746-8094
DOI:10.1016/j.bspc.2024.106131