Two-Stream Architecture with Contrastive and Self-Supervised Attention Feature Fusion for Error-related Potentials Classification

Error-related potentials (ErrPs) extracted from electroencephalographic signals hold potential for application in Brain-Machine Interfaces, in contexts such as robot teleoperation or shared control in assistive platforms. Due to difficulties in signal classification, in part caused by its non-statio...

Full description

Saved in:
Bibliographic Details
Published in:IEEE RO-MAN pp. 118 - 124
Main Authors: Garrote, Luis, Perdiz, Joao, Yasemin, Mine, Pires, Gabriel, Nunes, Urbano J.
Format: Conference Proceeding
Language:English
Published: IEEE 26.08.2024
Subjects:
Brain modeling
Contrastive learning
Convolutional neural networks
Deep architecture
Electric potential
Electroencephalography
Feature extraction
Noise measurement
Robots
Training
ISSN:1944-9437
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Error-related potentials (ErrPs) extracted from electroencephalographic signals hold potential for application in Brain-Machine Interfaces, in contexts such as robot teleoperation or shared control in assistive platforms. Due to difficulties in signal classification, in part caused by its non-stationary and noisy nature, their use has not been fully realized yet.This work proposes a new approach to ErrP classification based on a two-stream deep learning architecture with three training stages. Its first stage is a self-supervised autoencoder architecture with a multi-head attention layer providing relevant latent features. The second stage comprises a supervised contrastive learning approach considering two backbone networks, where one inherits weights from the first stage and the other is updated by considering the feature embeddings distribution. The final stage comprises supervised classification, where the two backbones are fused and used to classify the input EEG signal. At the end of the three stages, a data-driven two-stream ErrP model is obtained.Twenty-five variants of the proposed approach using the Deep Convolutional Network, Shallow Convolutional Network and EEGNet backbones were tested in an ablation study and benchmarked against a large number of classical classification methods, using data from the BNCI dataset intended to assess cross subject generalization capabilities. The proposed approach obtained the best results overall, highlighting the approach's capabilities in capturing relevant representations of the EEG signal.
AbstractList Error-related potentials (ErrPs) extracted from electroencephalographic signals hold potential for application in Brain-Machine Interfaces, in contexts such as robot teleoperation or shared control in assistive platforms. Due to difficulties in signal classification, in part caused by its non-stationary and noisy nature, their use has not been fully realized yet.This work proposes a new approach to ErrP classification based on a two-stream deep learning architecture with three training stages. Its first stage is a self-supervised autoencoder architecture with a multi-head attention layer providing relevant latent features. The second stage comprises a supervised contrastive learning approach considering two backbone networks, where one inherits weights from the first stage and the other is updated by considering the feature embeddings distribution. The final stage comprises supervised classification, where the two backbones are fused and used to classify the input EEG signal. At the end of the three stages, a data-driven two-stream ErrP model is obtained.Twenty-five variants of the proposed approach using the Deep Convolutional Network, Shallow Convolutional Network and EEGNet backbones were tested in an ablation study and benchmarked against a large number of classical classification methods, using data from the BNCI dataset intended to assess cross subject generalization capabilities. The proposed approach obtained the best results overall, highlighting the approach's capabilities in capturing relevant representations of the EEG signal.
Author Perdiz, Joao
Yasemin, Mine
Nunes, Urbano J.
Garrote, Luis
Pires, Gabriel
Author_xml – sequence: 1
  givenname: Luis
  surname: Garrote
  fullname: Garrote, Luis
  email: garrote@isr.uc.pt
  organization: University of Coimbra,Institute of Systems and Robotics,Department of Electrical and Computer Engineering,Portugal
– sequence: 2
  givenname: Joao
  surname: Perdiz
  fullname: Perdiz, Joao
  email: joao.perdiz@isr.uc.pt
  organization: University of Coimbra,Institute of Systems and Robotics,Department of Electrical and Computer Engineering,Portugal
– sequence: 3
  givenname: Mine
  surname: Yasemin
  fullname: Yasemin, Mine
  email: mine.yasemin@isr.uc.pt
  organization: University of Coimbra,Institute of Systems and Robotics,Department of Electrical and Computer Engineering,Portugal
– sequence: 4
  givenname: Gabriel
  surname: Pires
  fullname: Pires, Gabriel
  email: gpires@isr.uc.pt
  organization: University of Coimbra,Institute of Systems and Robotics,Department of Electrical and Computer Engineering,Portugal
– sequence: 5
  givenname: Urbano J.
  surname: Nunes
  fullname: Nunes, Urbano J.
  email: urbano@isr.uc.pt
  organization: University of Coimbra,Institute of Systems and Robotics,Department of Electrical and Computer Engineering,Portugal
BookMark eNo1kMFOwzAQRA0CiVL6BxzMB7jYsRPHx6hqAalQRMu52iRr1ShNKtul4sifkxbYy2pGb1aavSYXbdciIXeCj4Xg5v5twZ6Ll4yLLB8nPFFjwbUUSqszMjLa5DLlUqc8Sc7JQBilmFFSX5FRCB-8H6OE0OmAfK8OHVtGj7Clha82LmIV9x7pwcUNnXRt9BCi-0QKbU2X2Fi23O_Qf7qANS1ixDa6rqUzhFNstg9HaTtPp953nnlsIPboa3dCoQl00kAIzroKjtEbcml7F0d_e0jeZ9PV5JHNFw9Pk2LOnNBZZCopJa8MaJ2nNk0SDrrifVGRZVKXNs1lDgZAS1PVpbKlzKStUNq6hlIaDXJIbn_vOkRc77zbgv9a_39N_gBwEWb0
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/RO-MAN60168.2024.10731474
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
IEEE Proceedings Order Plan All Online (POP All Online) 1998-present by volume
IEEE Xplore All Conference Proceedings
IEEE Electronic Library (IEL)
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
Discipline Engineering
EISBN 9798350375022
EISSN 1944-9437
EndPage 124
ExternalDocumentID 10731474
Genre orig-research
GrantInformation_xml – fundername: Foundation for Science and Technology
  funderid: 10.13039/501100019370
GroupedDBID 6IE
6IF
6IH
6IK
6IL
6IN
AAJGR
AAWTH
ADZIZ
ALMA_UNASSIGNED_HOLDINGS
BEFXN
BFFAM
BGNUA
BKEBE
BPEOZ
CBEJK
CHZPO
IPLJI
OCL
RIE
RIL
RNS
ID FETCH-LOGICAL-i176t-42b30c9a7785f5220a7c097916637bf5838a9aa739cdb4fb363fce3fddab397a3
IEDL.DBID RIE
ISICitedReferencesCount 0
ISICitedReferencesURI http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001348918600018&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
IngestDate Wed Aug 27 03:01:26 EDT 2025
IsPeerReviewed false
IsScholarly true
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i176t-42b30c9a7785f5220a7c097916637bf5838a9aa739cdb4fb363fce3fddab397a3
PageCount 7
ParticipantIDs ieee_primary_10731474
PublicationCentury 2000
PublicationDate 2024-Aug.-26
PublicationDateYYYYMMDD 2024-08-26
PublicationDate_xml – month: 08
  year: 2024
  text: 2024-Aug.-26
  day: 26
PublicationDecade 2020
PublicationTitle IEEE RO-MAN
PublicationTitleAbbrev RO-MAN
PublicationYear 2024
Publisher IEEE
Publisher_xml – name: IEEE
SSID ssj0000941175
Score 2.2665093
Snippet Error-related potentials (ErrPs) extracted from electroencephalographic signals hold potential for application in Brain-Machine Interfaces, in contexts such as...
SourceID ieee
SourceType Publisher
StartPage 118
SubjectTerms Brain modeling
Contrastive learning
Convolutional neural networks
Deep architecture
Electric potential
Electroencephalography
Feature extraction
Noise measurement
Robots
Training
Title Two-Stream Architecture with Contrastive and Self-Supervised Attention Feature Fusion for Error-related Potentials Classification
URI https://ieeexplore.ieee.org/document/10731474
WOSCitedRecordID wos001348918600018&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA62iOjFV8U3Ebym7m7WZHMs0uJBa7EVeit5QkF3y3ZXz_5zM2mt9eDB27IkYckkM7Mz832D0LVLmTUZNyQTUhOAahLp7RCJNU2US7LI6tC15IH3-9l4LAZLsHrAwlhrQ_GZbcNjyOWbQtcQKvM3nNM45WkDNTjnC7DWKqDi_1OAdnILXS15NG-en8hjpw98I1DDlaTt7_m_OqkEQ9Lb_ecn7KHWDyQPD1bGZh9t2PwA7ayxCR6iz9FHQSDLLN9wZy0_gCHWioGGqpRz0G5Y5gYP7asjw3oGumJuDe5U1aLyEYNXCNN6NUTSsPdqcbcsi5IE3IsfOijCUH9wceipCdVGQcAt9NLrju7uybLDApnGnFVeNopGWkjOs1vnPbFIch0J7l1GRrlykFKVQkpOhTYqdYoy6rSlzhipvCMj6RFq5kVujxF2sQurMCu88nVUpFRrkzElWQRcyCeoBbs5mS1INCbfG3n6x_sztA0yg_Btws5Rsypre4E29Xs1nZeXQfRfDhSx4g
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LT8MwDI5gIB4XXkO8CRLXQNt0TXOc0KYhtjGxIe02pXlIk6CduhbO_HPi7ME4cOBWRWkUxant2v4-I3RrwkirmCkScyEJQDWJsHaI-JIGiQliT0vXtaTNut14OOS9OVjdYWG01q74TN_Bo8vlq0yWECqzXzijfsjCdbRRC8PAn8G1liEV-6cCxJNb6GbOpHn_8kw69S4wjkAVVxDeLVb41UvFmZLm3j83sY-qP6A83FuamwO0ptNDtLvCJ3iEvgafGYE8s3jH9ZUMAYZoKwYiqlxMQb9hkSrc12-G9MsJaIupVrheFLPaRwx-IbzWLCGWhq1fixt5nuXEIV_s1F7mptqri11XTag3ciKuotdmY_DQIvMeC2Tss6iw0kmoJ7lgLK4Z64t5gkmPM-s0RpQlBpKqggvBKJcqCU1CI2qkpkYpkVhXRtBjVEmzVJ8gbHzjVok0t-rXUB5SKVUcJSLygA35FFXhNEeTGY3GaHGQZ3-MX6Pt1qDTHrUfu0_naAfkB8HcILpAlSIv9SXalB_FeJpfuWvwDXd8tSk
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=IEEE+RO-MAN&rft.atitle=Two-Stream+Architecture+with+Contrastive+and+Self-Supervised+Attention+Feature+Fusion+for+Error-related+Potentials+Classification&rft.au=Garrote%2C+Luis&rft.au=Perdiz%2C+Joao&rft.au=Yasemin%2C+Mine&rft.au=Pires%2C+Gabriel&rft.date=2024-08-26&rft.pub=IEEE&rft.eissn=1944-9437&rft.spage=118&rft.epage=124&rft_id=info:doi/10.1109%2FRO-MAN60168.2024.10731474&rft.externalDocID=10731474