Two-Stream Architecture with Contrastive and Self-Supervised Attention Feature Fusion for Error-related Potentials Classification

Error-related potentials (ErrPs) extracted from electroencephalographic signals hold potential for application in Brain-Machine Interfaces, in contexts such as robot teleoperation or shared control in assistive platforms. Due to difficulties in signal classification, in part caused by its non-statio...

Full description

Saved in:
Bibliographic Details
Published in:IEEE RO-MAN pp. 118 - 124
Main Authors: Garrote, Luis, Perdiz, Joao, Yasemin, Mine, Pires, Gabriel, Nunes, Urbano J.
Format: Conference Proceeding
Language:English
Published: IEEE 26.08.2024
Subjects:
Brain modeling
Contrastive learning
Convolutional neural networks
Deep architecture
Electric potential
Electroencephalography
Feature extraction
Noise measurement
Robots
Training
ISSN:1944-9437
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Error-related potentials (ErrPs) extracted from electroencephalographic signals hold potential for application in Brain-Machine Interfaces, in contexts such as robot teleoperation or shared control in assistive platforms. Due to difficulties in signal classification, in part caused by its non-stationary and noisy nature, their use has not been fully realized yet.This work proposes a new approach to ErrP classification based on a two-stream deep learning architecture with three training stages. Its first stage is a self-supervised autoencoder architecture with a multi-head attention layer providing relevant latent features. The second stage comprises a supervised contrastive learning approach considering two backbone networks, where one inherits weights from the first stage and the other is updated by considering the feature embeddings distribution. The final stage comprises supervised classification, where the two backbones are fused and used to classify the input EEG signal. At the end of the three stages, a data-driven two-stream ErrP model is obtained.Twenty-five variants of the proposed approach using the Deep Convolutional Network, Shallow Convolutional Network and EEGNet backbones were tested in an ablation study and benchmarked against a large number of classical classification methods, using data from the BNCI dataset intended to assess cross subject generalization capabilities. The proposed approach obtained the best results overall, highlighting the approach's capabilities in capturing relevant representations of the EEG signal.
ISSN:1944-9437
DOI:10.1109/RO-MAN60168.2024.10731474