MSLTE: multiple self-supervised learning tasks for enhancing EEG emotion recognition

. The instability of the EEG acquisition devices may lead to information loss in the channels or frequency bands of the collected EEG. This phenomenon may be ignored in available models, which leads to the overfitting and low generalization of the model. . Multiple self-supervised learning tasks are...

Full description

Saved in:

Bibliographic Details
Published in:	Journal of neural engineering Vol. 21; no. 2
Main Authors:	Li, Guangqiang, Chen, Ning, Niu, Yixiang, Xu, Zhangyong, Dong, Yuxuan, Jin, Jing, Zhu, Hongqin
Format:	Journal Article
Language:	English
Published:	England 01.04.2024
Subjects:	Datasets as Topic Electroencephalography Emotions Humans Supervised Machine Learning - standards multi-task learning EEG emotion recognition graph autoencoder weight sharing mask-based self-supervised learning
ISSN:	1741-2552, 1741-2552
Online Access:	Get more information
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	. The instability of the EEG acquisition devices may lead to information loss in the channels or frequency bands of the collected EEG. This phenomenon may be ignored in available models, which leads to the overfitting and low generalization of the model. . Multiple self-supervised learning tasks are introduced in the proposed model to enhance the generalization of EEG emotion recognition and reduce the overfitting problem to some extent. Firstly, channel masking and frequency masking are introduced to simulate the information loss in certain channels and frequency bands resulting from the instability of EEG, and two self-supervised learning-based feature reconstruction tasks combining masked graph autoencoders (GAE) are constructed to enhance the generalization of the shared encoder. Secondly, to take full advantage of the complementary information contained in these two self-supervised learning tasks to ensure the reliability of feature reconstruction, a weight sharing (WS) mechanism is introduced between the two graph decoders. Thirdly, an adaptive weight multi-task loss (AWML) strategy based on homoscedastic uncertainty is adopted to combine the supervised learning loss and the two self-supervised learning losses to enhance the performance further. . Experimental results on SEED, SEED-V, and DEAP datasets demonstrate that: (i) Generally, the proposed model achieves higher averaged emotion classification accuracy than various baselines included in both subject-dependent and subject-independent scenarios. (ii) Each key module contributes to the performance enhancement of the proposed model. (iii) It achieves higher training efficiency, and significantly lower model size and computational complexity than the state-of-the-art (SOTA) multi-task-based model. (iv) The performances of the proposed model are less influenced by the key parameters. . The introduction of the self-supervised learning task helps to enhance the generalization of the EEG emotion recognition model and eliminate overfitting to some extent, which can be modified to be applied in other EEG-based classification tasks.
AbstractList	Objective. The instability of the EEG acquisition devices may lead to information loss in the channels or frequency bands of the collected EEG. This phenomenon may be ignored in available models, which leads to the overfitting and low generalization of the model.Approach. Multiple self-supervised learning tasks are introduced in the proposed model to enhance the generalization of EEG emotion recognition and reduce the overfitting problem to some extent. Firstly, channel masking and frequency masking are introduced to simulate the information loss in certain channels and frequency bands resulting from the instability of EEG, and two self-supervised learning-based feature reconstruction tasks combining masked graph autoencoders (GAE) are constructed to enhance the generalization of the shared encoder. Secondly, to take full advantage of the complementary information contained in these two self-supervised learning tasks to ensure the reliability of feature reconstruction, a weight sharing (WS) mechanism is introduced between the two graph decoders. Thirdly, an adaptive weight multi-task loss (AWML) strategy based on homoscedastic uncertainty is adopted to combine the supervised learning loss and the two self-supervised learning losses to enhance the performance further.Main results. Experimental results on SEED, SEED-V, and DEAP datasets demonstrate that: (i) Generally, the proposed model achieves higher averaged emotion classification accuracy than various baselines included in both subject-dependent and subject-independent scenarios. (ii) Each key module contributes to the performance enhancement of the proposed model. (iii) It achieves higher training efficiency, and significantly lower model size and computational complexity than the state-of-the-art (SOTA) multi-task-based model. (iv) The performances of the proposed model are less influenced by the key parameters.Significance. The introduction of the self-supervised learning task helps to enhance the generalization of the EEG emotion recognition model and eliminate overfitting to some extent, which can be modified to be applied in other EEG-based classification tasks.Objective. The instability of the EEG acquisition devices may lead to information loss in the channels or frequency bands of the collected EEG. This phenomenon may be ignored in available models, which leads to the overfitting and low generalization of the model.Approach. Multiple self-supervised learning tasks are introduced in the proposed model to enhance the generalization of EEG emotion recognition and reduce the overfitting problem to some extent. Firstly, channel masking and frequency masking are introduced to simulate the information loss in certain channels and frequency bands resulting from the instability of EEG, and two self-supervised learning-based feature reconstruction tasks combining masked graph autoencoders (GAE) are constructed to enhance the generalization of the shared encoder. Secondly, to take full advantage of the complementary information contained in these two self-supervised learning tasks to ensure the reliability of feature reconstruction, a weight sharing (WS) mechanism is introduced between the two graph decoders. Thirdly, an adaptive weight multi-task loss (AWML) strategy based on homoscedastic uncertainty is adopted to combine the supervised learning loss and the two self-supervised learning losses to enhance the performance further.Main results. Experimental results on SEED, SEED-V, and DEAP datasets demonstrate that: (i) Generally, the proposed model achieves higher averaged emotion classification accuracy than various baselines included in both subject-dependent and subject-independent scenarios. (ii) Each key module contributes to the performance enhancement of the proposed model. (iii) It achieves higher training efficiency, and significantly lower model size and computational complexity than the state-of-the-art (SOTA) multi-task-based model. (iv) The performances of the proposed model are less influenced by the key parameters.Significance. The introduction of the self-supervised learning task helps to enhance the generalization of the EEG emotion recognition model and eliminate overfitting to some extent, which can be modified to be applied in other EEG-based classification tasks. . The instability of the EEG acquisition devices may lead to information loss in the channels or frequency bands of the collected EEG. This phenomenon may be ignored in available models, which leads to the overfitting and low generalization of the model. . Multiple self-supervised learning tasks are introduced in the proposed model to enhance the generalization of EEG emotion recognition and reduce the overfitting problem to some extent. Firstly, channel masking and frequency masking are introduced to simulate the information loss in certain channels and frequency bands resulting from the instability of EEG, and two self-supervised learning-based feature reconstruction tasks combining masked graph autoencoders (GAE) are constructed to enhance the generalization of the shared encoder. Secondly, to take full advantage of the complementary information contained in these two self-supervised learning tasks to ensure the reliability of feature reconstruction, a weight sharing (WS) mechanism is introduced between the two graph decoders. Thirdly, an adaptive weight multi-task loss (AWML) strategy based on homoscedastic uncertainty is adopted to combine the supervised learning loss and the two self-supervised learning losses to enhance the performance further. . Experimental results on SEED, SEED-V, and DEAP datasets demonstrate that: (i) Generally, the proposed model achieves higher averaged emotion classification accuracy than various baselines included in both subject-dependent and subject-independent scenarios. (ii) Each key module contributes to the performance enhancement of the proposed model. (iii) It achieves higher training efficiency, and significantly lower model size and computational complexity than the state-of-the-art (SOTA) multi-task-based model. (iv) The performances of the proposed model are less influenced by the key parameters. . The introduction of the self-supervised learning task helps to enhance the generalization of the EEG emotion recognition model and eliminate overfitting to some extent, which can be modified to be applied in other EEG-based classification tasks.
Author	Chen, Ning Niu, Yixiang Li, Guangqiang Zhu, Hongqin Jin, Jing Dong, Yuxuan Xu, Zhangyong
Author_xml	– sequence: 1 givenname: Guangqiang surname: Li fullname: Li, Guangqiang organization: School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, People's Republic of China – sequence: 2 givenname: Ning orcidid: 0000-0002-8171-267X surname: Chen fullname: Chen, Ning organization: School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, People's Republic of China – sequence: 3 givenname: Yixiang surname: Niu fullname: Niu, Yixiang organization: School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, People's Republic of China – sequence: 4 givenname: Zhangyong surname: Xu fullname: Xu, Zhangyong organization: School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, People's Republic of China – sequence: 5 givenname: Yuxuan surname: Dong fullname: Dong, Yuxuan organization: School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, People's Republic of China – sequence: 6 givenname: Jing surname: Jin fullname: Jin, Jing organization: School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, People's Republic of China – sequence: 7 givenname: Hongqin surname: Zhu fullname: Zhu, Hongqin organization: School of Information Science and Engineering, East China University of Science and Technology, Shanghai 200237, People's Republic of China
BackLink	https://www.ncbi.nlm.nih.gov/pubmed/38588700$$D View this record in MEDLINE/PubMed
BookMark	eNpNkDtPwzAYRS1URB-wMyGPLKF-Jg4bqkJBKmKgzJHtfCkGxwlxgsS_h4qCmO7R1dUdzhxNQhsAoXNKrihRakkzQRMmJVvqilumjtDsr5r84ymax_hKCKdZTk7QlCupVEbIDG0fnjbb4ho3ox9c5wFH8HUSxw76Dxehwh50H1zY4UHHt4jrtscQXnSw-64o1hiadnBtwD3Ydhfcnk_Rca19hLNDLtDzbbFd3SWbx_X96maTWJ6pIUlNVpNcVDrnkltOtREcmKTCCG2sVallvBLWCCtMmnJRgyC0zqWVhlkFwBbo8ue369v3EeJQNi5a8F4HaMdYcsIlyWT-bWCBLg7T0TRQlV3vGt1_lr8m2BeA7WKr
CitedBy_id	crossref_primary_10_1016_j_knosys_2025_113368 crossref_primary_10_1016_j_neunet_2025_107614 crossref_primary_10_1016_j_neunet_2025_107363 crossref_primary_10_1109_TIM_2025_3590828 crossref_primary_10_1016_j_knosys_2025_114364
ContentType	Journal Article
Copyright	2024 IOP Publishing Ltd.
Copyright_xml	– notice: 2024 IOP Publishing Ltd.
DBID	CGR CUY CVF ECM EIF NPM 7X8
DOI	10.1088/1741-2552/ad3c28
DatabaseName	Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed MEDLINE - Academic
DatabaseTitle	MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) MEDLINE - Academic
DatabaseTitleList	MEDLINE - Academic MEDLINE
Database_xml	– sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database
DeliveryMethod	no_fulltext_linktorsrc
Discipline	Anatomy & Physiology
EISSN	1741-2552
ExternalDocumentID	38588700
Genre	Research Support, Non-U.S. Gov't Journal Article
GroupedDBID	--- 1JI 4.4 53G 5B3 5GY 5VS 5ZH 7.M 7.Q AAGCD AAJIO AAJKP AATNI ABHWH ABJNI ABQJV ABVAM ACAFW ACGFS ACHIP ADEQX AEFHF AENEX AFYNE AKPSB ALMA_UNASSIGNED_HOLDINGS AOAED ASPBG ATQHT AVWKF AZFZN CEBXE CGR CJUJL CRLBU CS3 CUY CVF DU5 EBS ECM EDWGO EIF EMSAF EPQRW EQZZN F5P IHE IJHAN IOP IZVLO KOT LAP N5L N9A NPM P2P PJBAE RIN RO9 ROL RPA SY9 W28 XPP 7X8 AEINN
ID	FETCH-LOGICAL-c378t-6b7f094da9353c31ab43e2514b4abcc86c23d4cb4c4b6634fe401f95c5b2c8ee2
IEDL.DBID	7X8
ISICitedReferencesCount	6
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001204292900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1741-2552
IngestDate	Fri Sep 05 13:49:06 EDT 2025 Sun Jun 15 01:31:12 EDT 2025
IsDoiOpenAccess	false
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	2
Keywords	multi-task learning EEG emotion recognition graph autoencoder weight sharing mask-based self-supervised learning
Language	English
License	2024 IOP Publishing Ltd.
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c378t-6b7f094da9353c31ab43e2514b4abcc86c23d4cb4c4b6634fe401f95c5b2c8ee2
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23
ORCID	0000-0002-8171-267X
OpenAccessLink	https://doi.org/10.1088/1741-2552/ad3c28
PMID	38588700
PQID	3035075955
PQPubID	23479
ParticipantIDs	proquest_miscellaneous_3035075955 pubmed_primary_38588700
PublicationCentury	2000
PublicationDate	2024-04-01
PublicationDateYYYYMMDD	2024-04-01
PublicationDate_xml	– month: 04 year: 2024 text: 2024-04-01 day: 01
PublicationDecade	2020
PublicationPlace	England
PublicationPlace_xml	– name: England
PublicationTitle	Journal of neural engineering
PublicationTitleAlternate	J Neural Eng
PublicationYear	2024
SSID	ssj0031790
Score	2.4401755
Snippet	. The instability of the EEG acquisition devices may lead to information loss in the channels or frequency bands of the collected EEG. This phenomenon may be... Objective. The instability of the EEG acquisition devices may lead to information loss in the channels or frequency bands of the collected EEG. This phenomenon...
SourceID	proquest pubmed
SourceType	Aggregation Database Index Database
SubjectTerms	Datasets as Topic Electroencephalography Emotions Humans Supervised Machine Learning - standards
Title	MSLTE: multiple self-supervised learning tasks for enhancing EEG emotion recognition
URI	https://www.ncbi.nlm.nih.gov/pubmed/38588700 https://www.proquest.com/docview/3035075955
Volume	21
WOSCitedRecordID	wos001204292900001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText
inHoldings	1
isFullTextHit
isPrint
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8QwEA7qevDia32sLyKIt7Bt0jatF1mk6sFdFlxhbyWvqqjtanYF_72TtrueBMFLoYFASCfTb-abzIfQGTXUtfHWRFNfEHB4PomZgNdQ8JgpanikK7EJPhjE43EybBJutimrnPvEylHrUrkceZc5CoyHSRheTt6JU41y7GojobGMWgygjLNqPl6wCMx1n6ovRPoEoDNtaEo4WN3FWFdoWFL8O8CsfjTXG_9d4iZabyAm7tU2sYWWTLGN2r0Cwuu3L3yOq6LPKpveRqP-_d0ovcDzukJszWtO7GziXIg1GjeqEo94KuyLxQBxsSmeXJMOGEvTG2xqHSC8qEQqix30cJ2Orm5JI7RAFOPxlESS5xDmaZGwkCnmCxkwA8AnkIGQSsWRokwHSgYqkIBQgtxAVJYnoQolVbExdBetFGVh9hH2eJRQX3uecTdudSSU8LgMZaK9nCmPd9DpfO8yMGTHTojClDOb_exeB-3VHyCb1B03MsdegmPxDv4w-xCtUQAedXXNEWrlcIzNMVpVn9Nn-3FSWQg8B8P-Nx_MxbE
linkProvider	ProQuest
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=MSLTE%3A+multiple+self-supervised+learning+tasks+for+enhancing+EEG+emotion+recognition&rft.jtitle=Journal+of+neural+engineering&rft.au=Li%2C+Guangqiang&rft.au=Chen%2C+Ning&rft.au=Niu%2C+Yixiang&rft.au=Xu%2C+Zhangyong&rft.date=2024-04-01&rft.issn=1741-2552&rft.eissn=1741-2552&rft.volume=21&rft.issue=2&rft_id=info:doi/10.1088%2F1741-2552%2Fad3c28&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1741-2552&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1741-2552&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1741-2552&client=summon