A multi-task hybrid emotion recognition network based on EEG signals

Electroencephalogram (EEG)-based emotion recognition is getting a lot of attention in Brain–Computer Interface (BCI) research. However, the recognition model of cross-subject emotions requires effectively represented EEG data to capture diverse features corresponding to the spatial, frequency and te...

Full description

Saved in:
Bibliographic Details
Published in:Biomedical signal processing and control Vol. 86; p. 105136
Main Authors: Zhou, Qiaoli, Shi, Chi, Du, Qiang, Ke, Li
Format: Journal Article
Language:English
Published: Elsevier Ltd 01.09.2023
Subjects:
Brain map
Deep learning
EEG
Emotion recognition
Multi-task learning
Brain map
Deep learning
Emotion recognition
Multi-task learning
EEG
ISSN:1746-8094, 1746-8108
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Electroencephalogram (EEG)-based emotion recognition is getting a lot of attention in Brain–Computer Interface (BCI) research. However, the recognition model of cross-subject emotions requires effectively represented EEG data to capture diverse features corresponding to the spatial, frequency and temporal of information. In contrast, by calculating Power Spectral Density (PSD), Differential Entropy (DE), and the spaced interpolation algorithm, brain map construction has the ability to enhance the quality of frequency and spatial data features. In addition, DEAP is a dimensional dataset. We used the multi-task learning method while considering the characteristics of the dimensional dataset. Therefore, we proposed a novel hybrid emotion recognition deep learning network called the multi-task hybrid emotion recognition network(MTHNet), which fuses Bi-LSTM(Bidirectional long short-term memory) and improved UNet. In the hybrid network, improved UNet is utilized to extract spatial features. Then Bi-LSTM cells are applied to extract temporal features and memorize the change of sequential data in forward and backward directions. The proposed method is applied to the well-known 5 public databases and has given outperforming results than the current state of the art. •The proposed method can capture heterogeneous features representing spatial, temporal and frequency of EEG.•The improved UNet address model degradation during cross-subject training.•The size of the time window affects the connection between various EEG segments.•The spatial interpolation algorithm makes the brain map less affected by electrode position.•Conversion method from single-task to multi-task is proposed for single task or discrete datasets.
ISSN:1746-8094
1746-8108
DOI:10.1016/j.bspc.2023.105136