Combining multi-scale convolutional neural network and Transformers for EEG-Based RSVP detection

Rapid serial visual presentation (RSVP) is an effective brain-computer interface (BCI) technique for recognizing target objects. Decoding the subject's intention from the single-trial electroencephalogram (EEG) signal through a decoding algorithm is the key to RSVP-based BCI. The unavoidable no...

Full description

Saved in:
Bibliographic Details
Published in:2022 37th Youth Academic Annual Conference of Chinese Association of Automation (YAC) pp. 426 - 431
Main Authors: Lu, Gai, Zhang, Yifan, Chu, Xingxing, Liu, Yingxin, Yu, Yang
Format: Conference Proceeding
Language:English
Published: IEEE 19.11.2022
Subjects:
Benchmark testing
Brain modeling
EEG decoding algorithm
electroencephalogram(EEG)
Electroencephalography
Feature extraction
multi-scale
Target recognition
Transformers
Visualization
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Abstract Rapid serial visual presentation (RSVP) is an effective brain-computer interface (BCI) technique for recognizing target objects. Decoding the subject's intention from the single-trial electroencephalogram (EEG) signal through a decoding algorithm is the key to RSVP-based BCI. The unavoidable noise and variability between trials in EEG signals lead to low accuracy of EEG-based RSVP detection and low universality of the model. It is necessary to develop an EEG decoding algorithm with robust generalization ability and high recognition accuracy. In this study, we proposed a novel end-to-end model architecture that combines multi-scale spatiotemporal convolutional neural network (CNN) and Transformers. Specifically, the multi-scale CNN is used to capture spatiotemporal features at different scales, while the Transformers are used to extract the most discriminative global information. Experimental results on the RSVP-based benchmark datasets show that the proposed method in this study can achieve higher recognition accuracy compared to the other three advanced methods in both cross-subject and within-subject experiments. The results of fine-tuning experiments using pre-trained models on a new subject show that better results can be obtained in single-subject experiments using only a small amount of data. The experimental results validate the effectiveness of our method and provide a new idea for constructing a feature extraction method with better generalization capability for RSVP-based BCI.
AbstractList Rapid serial visual presentation (RSVP) is an effective brain-computer interface (BCI) technique for recognizing target objects. Decoding the subject's intention from the single-trial electroencephalogram (EEG) signal through a decoding algorithm is the key to RSVP-based BCI. The unavoidable noise and variability between trials in EEG signals lead to low accuracy of EEG-based RSVP detection and low universality of the model. It is necessary to develop an EEG decoding algorithm with robust generalization ability and high recognition accuracy. In this study, we proposed a novel end-to-end model architecture that combines multi-scale spatiotemporal convolutional neural network (CNN) and Transformers. Specifically, the multi-scale CNN is used to capture spatiotemporal features at different scales, while the Transformers are used to extract the most discriminative global information. Experimental results on the RSVP-based benchmark datasets show that the proposed method in this study can achieve higher recognition accuracy compared to the other three advanced methods in both cross-subject and within-subject experiments. The results of fine-tuning experiments using pre-trained models on a new subject show that better results can be obtained in single-subject experiments using only a small amount of data. The experimental results validate the effectiveness of our method and provide a new idea for constructing a feature extraction method with better generalization capability for RSVP-based BCI.
Author Liu, Yingxin
Yu, Yang
Zhang, Yifan
Lu, Gai
Chu, Xingxing
Author_xml – sequence: 1
  givenname: Gai
  surname: Lu
  fullname: Lu, Gai
  organization: National University of Defense Technology,College of Intelligence Science and Technology,ChangSha,China
– sequence: 2
  givenname: Yifan
  surname: Zhang
  fullname: Zhang, Yifan
  organization: National University of Defense Technology,College of Intelligence Science and Technology,ChangSha,China
– sequence: 3
  givenname: Xingxing
  surname: Chu
  fullname: Chu, Xingxing
  organization: National University of Defense Technology,College of Intelligence Science and Technology,ChangSha,China
– sequence: 4
  givenname: Yingxin
  surname: Liu
  fullname: Liu, Yingxin
  organization: National University of Defense Technology,College of Intelligence Science and Technology,ChangSha,China
– sequence: 5
  givenname: Yang
  surname: Yu
  fullname: Yu, Yang
  email: yuyangnudt@hotmail.com
  organization: National University of Defense Technology,College of Intelligence Science and Technology,ChangSha,China
BookMark eNo1z81KxDAYheEIutBx7kAkN9CanzZNlmOpozCgaBFcjV-arxJsE0k7infv-Ld6VufAe0IOQwxIyDlnOefMXDyt6rISWuSCCZFzxoRUWh-Qpak0V6osVCmVOibPdRytDz680HE3zD6bOhiQdjG8x2E3-xhgoAF36Yf5I6ZXCsHRNkGY-phGTBPdS5tmnV3ChI7ePzzeUYczdt_zU3LUwzDh8s8Faa-atr7ONrfrm3q1yXxhygxBGsOYFZZZVUJfsc6CRqmgxIoD641E13HlLBjBNYN9hitAgeIaCwFyQc5-bz0ibt-SHyF9bv-75Rd40VOc
ContentType Conference Proceeding
DBID 6IE
6IL
CBEJK
RIE
RIL
DOI 10.1109/YAC57282.2022.10023688
DatabaseName IEEE Electronic Library (IEL) Conference Proceedings
Accès Toulouse INP et ENVT - IEEE Xplore POP ALL
IEEE Xplore All Conference Proceedings
IEEE Xplore
IEEE Proceedings Order Plans (POP All) 1998-Present
DatabaseTitleList
Database_xml – sequence: 1
  dbid: RIE
  name: IEEE Electronic Library (IEL)
  url: https://ieeexplore.ieee.org/
  sourceTypes: Publisher
DeliveryMethod fulltext_linktorsrc
EISBN 9781665465366
1665465360
EndPage 431
ExternalDocumentID 10023688
Genre orig-research
GrantInformation_xml – fundername: Technology Development
  funderid: 10.13039/100006180
– fundername: National Natural Science Foundation of China
  funderid: 10.13039/501100001809
GroupedDBID 6IE
6IL
CBEJK
RIE
RIL
ID FETCH-LOGICAL-i495-ea39900b2b0b65af70cba8e36a5e71a0f93edc16dba92180a978d4a6a618e42a3
IEDL.DBID RIE
IngestDate Thu Jan 18 11:14:52 EST 2024
IsPeerReviewed false
IsScholarly false
Language English
LinkModel DirectLink
MergedId FETCHMERGED-LOGICAL-i495-ea39900b2b0b65af70cba8e36a5e71a0f93edc16dba92180a978d4a6a618e42a3
PageCount 6
ParticipantIDs ieee_primary_10023688
PublicationCentury 2000
PublicationDate 2022-Nov.-19
PublicationDateYYYYMMDD 2022-11-19
PublicationDate_xml – month: 11
  year: 2022
  text: 2022-Nov.-19
  day: 19
PublicationDecade 2020
PublicationTitle 2022 37th Youth Academic Annual Conference of Chinese Association of Automation (YAC)
PublicationTitleAbbrev YAC
PublicationYear 2022
Publisher IEEE
Publisher_xml – name: IEEE
Score 1.8135515
Snippet Rapid serial visual presentation (RSVP) is an effective brain-computer interface (BCI) technique for recognizing target objects. Decoding the subject's...
SourceID ieee
SourceType Publisher
StartPage 426
SubjectTerms Benchmark testing
Brain modeling
EEG decoding algorithm
electroencephalogram(EEG)
Electroencephalography
Feature extraction
multi-scale
Target recognition
Transformers
Visualization
Title Combining multi-scale convolutional neural network and Transformers for EEG-Based RSVP detection
URI https://ieeexplore.ieee.org/document/10023688
hasFullText 1
inHoldings 1
isFullTextHit
isPrint
link http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV1LSwMxEA62ePCkYsU3OXhNm2R3k81RS6unUrRIPdU8ZqEgW-nD3-8k3SoePHjKErIszCR8M5vvmyHkVvkyGB7wIOksZ7kSjpXSOeZs6RFQdKyYkppN6NGonE7NuBGrJy0MACTyGXTjY7rLDwu_ib_KeiLVOy_LFmlprbZirUb1K7jpvd71C40pBGZ9UnZ3i3-1TUmoMTz85_eOSOdHf0fH38hyTPagPiFveHRdaudAEwuQrdC8QCNtvNk-9p3G8pRpSORuautAJ7vQFAM9iiMdDB7YPYJXoE_PL2MaYJ3oWHWHTIaDSf-RNf0R2BzTGgYWgwvOnXTcqcJWmnu0MGTKFqCF5ZXJIHihgrMGgZxbTBhDbpVVooRc2uyUtOtFDWeESlVEt0jvlc_xbeuCqIzNOUiX-yo7J51ondnHtgLGbGeYiz_mL8lB9EHU7AlzRdrr5Qauyb7_XM9Xy5vkty_5iZu7
linkProvider IEEE
linkToHtml http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwlV3PS8MwFA46BT2pOPG3OXjNlqRp2h51bE6cY2iReZr5VRhIJ1vn3-9L1ikePHhKCSmF9xK-95rvew-ha2lSm1ELBymJBBGSaZJyrYlWqQFASXzFlNBsIhkO0_E4G9Vi9aCFcc4F8plr-cdwl29nZul_lbVZqHeepptoKxaC05Vcq9b9Mpq1X286cQJJBOR9nLfWy381Tgm40dv75xf3UfNHgYdH39hygDZceYje4PDq0NABBx4gWYCBHfbE8XoDqXfsC1SGIdC7sSotztfBKYR6GEbc7d6RW4Avi5-eX0bYuioQssomynvdvNMndYcEMoXEhjgF4QWlmmuqZayKhBqwsYukil3CFC2yyFnDpNUqAyinClJGK5RUkqVOcBUdoUY5K90xwlzG3jHcGGkEvK20ZUWmBHVcC1NEJ6jprTP5WNXAmKwNc_rH_BXa6eePg8ngfvhwhna9P7yCj2XnqFHNl-4CbZvParqYXwYffgE_-p8C
openUrl ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Abook&rft.genre=proceeding&rft.title=2022+37th+Youth+Academic+Annual+Conference+of+Chinese+Association+of+Automation+%28YAC%29&rft.atitle=Combining+multi-scale+convolutional+neural+network+and+Transformers+for+EEG-Based+RSVP+detection&rft.au=Lu%2C+Gai&rft.au=Zhang%2C+Yifan&rft.au=Chu%2C+Xingxing&rft.au=Liu%2C+Yingxin&rft.date=2022-11-19&rft.pub=IEEE&rft.spage=426&rft.epage=431&rft_id=info:doi/10.1109%2FYAC57282.2022.10023688&rft.externalDocID=10023688