Multi-Modal Domain Adaptation Variational Autoencoder for EEG-Based Emotion Recognition

Traditional electroencephalograph (EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject, which restricts the application of the affective brain computer interface (BCI) in practice. We attempt to use the multi-modal data from the past...

Celý popis

Uloženo v:
Podrobná bibliografie
Vydáno v:IEEE/CAA journal of automatica sinica Ročník 9; číslo 9; s. 1612 - 1626
Hlavní autoři: Wang, Yixin, Qiu, Shuang, Li, Dan, Du, Changde, Lu, Bao-Liang, He, Huiguang
Médium: Journal Article
Jazyk:angličtina
Vydáno: Piscataway Chinese Association of Automation (CAA) 01.09.2022
The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
Témata:
Adaptation models
Brain modeling
Calibration
Cycle-consistency
Data models
domain adaptation
Domains
electroencephalograph (EEG)
Electroencephalography
Emotion recognition
Emotions
Human-computer interface
Image reconstruction
Knowledge management
Modal data
multi modality
Regularization
Representations
variational autoencoder
ISSN:2329-9266, 2329-9274
On-line přístup:Získat plný text
Tagy: Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
Popis
Shrnutí:Traditional electroencephalograph (EEG)-based emotion recognition requires a large number of calibration samples to build a model for a specific subject, which restricts the application of the affective brain computer interface (BCI) in practice. We attempt to use the multi-modal data from the past session to realize emotion recognition in the case of a small amount of calibration samples. To solve this problem, we propose a multi-modal domain adaptive variational autoencoder (MMDA-VAE) method, which learns shared cross-domain latent representations of the multi-modal data. Our method builds a multi-modal variational autoencoder (MVAE) to project the data of multiple modalities into a common space. Through adversarial learning and cycle-consistency regularization, our method can reduce the distribution difference of each domain on the shared latent representation layer and realize the transfer of knowledge. Extensive experiments are conducted on two public datasets, SEED and SEED-IV, and the results show the superiority of our proposed method. Our work can effectively improve the performance of emotion recognition with a small amount of labelled multi-modal data.
Bibliografie:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 14
ISSN:2329-9266
2329-9274
DOI:10.1109/JAS.2022.105515