Data augmentation for self-paced motor imagery classification with C-LSTM
Objective. Brain-computer interfaces (BCI) are becoming important tools for assistive technology, particularly through the use of motor imagery (MI) for aiding task completion. However, most existing methods of MI classification have been applied in a trial-wise fashion, with window sizes of approxi...
Uložené v:
| Vydané v: | Journal of neural engineering Ročník 17; číslo 1; s. 016041 - 16055 |
|---|---|
| Hlavní autori: | , |
| Médium: | Journal Article |
| Jazyk: | English |
| Vydavateľské údaje: |
England
IOP Publishing
31.01.2020
|
| Predmet: | |
| ISSN: | 1741-2560, 1741-2552, 1741-2552 |
| On-line prístup: | Získať plný text |
| Tagy: |
Pridať tag
Žiadne tagy, Buďte prvý, kto otaguje tento záznam!
|
| Abstract | Objective. Brain-computer interfaces (BCI) are becoming important tools for assistive technology, particularly through the use of motor imagery (MI) for aiding task completion. However, most existing methods of MI classification have been applied in a trial-wise fashion, with window sizes of approximately 2 s or more. Application of this type of classifier could cause a delay when switching between MI events. Approach. In this study, state-of-the-art classification methods for motor imagery are assessed offline with considerations for real-time and self-paced control, and a convolutional long-short term memory (C-LSTM) network based on filter bank common spatial patterns (FBCSP) is proposed. In addition, the effects of several methods of data augmentation on different classifiers are explored. Main results. The results of this study show that the proposed network achieves adequate results in distinguishing between different control classes, but both considered deep learning models are still less reliable than a Riemannian MDM classifier. In addition, controlled skewing of the data and the explored data augmentation methods improved the average overall accuracy of the classifiers by 14.0% and 5.3%, respectively. Significance. This manuscript is among the first to attempt combining convolutional and recurrent neural network layers for the purpose of MI classification, and is also one of the first to provide an in-depth comparison of various data augmentation methods for MI classification. In addition, all of these methods are applied on smaller windows of data and with consideration to ambient data, which provides a more realistic test bed for real-time and self-paced control. |
|---|---|
| AbstractList | Objective. Brain-computer interfaces (BCI) are becoming important tools for assistive technology, particularly through the use of motor imagery (MI) for aiding task completion. However, most existing methods of MI classification have been applied in a trial-wise fashion, with window sizes of approximately 2 s or more. Application of this type of classifier could cause a delay when switching between MI events. Approach. In this study, state-of-the-art classification methods for motor imagery are assessed offline with considerations for real-time and self-paced control, and a convolutional long-short term memory (C-LSTM) network based on filter bank common spatial patterns (FBCSP) is proposed. In addition, the effects of several methods of data augmentation on different classifiers are explored. Main results. The results of this study show that the proposed network achieves adequate results in distinguishing between different control classes, but both considered deep learning models are still less reliable than a Riemannian MDM classifier. In addition, controlled skewing of the data and the explored data augmentation methods improved the average overall accuracy of the classifiers by 14.0% and 5.3%, respectively. Significance. This manuscript is among the first to attempt combining convolutional and recurrent neural network layers for the purpose of MI classification, and is also one of the first to provide an in-depth comparison of various data augmentation methods for MI classification. In addition, all of these methods are applied on smaller windows of data and with consideration to ambient data, which provides a more realistic test bed for real-time and self-paced control. Brain-computer interfaces (BCI) are becoming important tools for assistive technology, particularly through the use of motor imagery (MI) for aiding task completion. However, most existing methods of MI classification have been applied in a trial-wise fashion, with window sizes of approximately 2 s or more. Application of this type of classifier could cause a delay when switching between MI events. In this study, state-of-the-art classification methods for motor imagery are assessed offline with considerations for real-time and self-paced control, and a convolutional long-short term memory (C-LSTM) network based on filter bank common spatial patterns (FBCSP) is proposed. In addition, the effects of several methods of data augmentation on different classifiers are explored. The results of this study show that the proposed network achieves adequate results in distinguishing between different control classes, but both considered deep learning models are still less reliable than a Riemannian MDM classifier. In addition, controlled skewing of the data and the explored data augmentation methods improved the average overall accuracy of the classifiers by 14.0% and 5.3%, respectively. This manuscript is among the first to attempt combining convolutional and recurrent neural network layers for the purpose of MI classification, and is also one of the first to provide an in-depth comparison of various data augmentation methods for MI classification. In addition, all of these methods are applied on smaller windows of data and with consideration to ambient data, which provides a more realistic test bed for real-time and self-paced control. Brain-computer interfaces (BCI) are becoming important tools for assistive technology, particularly through the use of motor imagery (MI) for aiding task completion. However, most existing methods of MI classification have been applied in a trial-wise fashion, with window sizes of approximately 2 s or more. Application of this type of classifier could cause a delay when switching between MI events.OBJECTIVEBrain-computer interfaces (BCI) are becoming important tools for assistive technology, particularly through the use of motor imagery (MI) for aiding task completion. However, most existing methods of MI classification have been applied in a trial-wise fashion, with window sizes of approximately 2 s or more. Application of this type of classifier could cause a delay when switching between MI events.In this study, state-of-the-art classification methods for motor imagery are assessed offline with considerations for real-time and self-paced control, and a convolutional long-short term memory (C-LSTM) network based on filter bank common spatial patterns (FBCSP) is proposed. In addition, the effects of several methods of data augmentation on different classifiers are explored.APPROACHIn this study, state-of-the-art classification methods for motor imagery are assessed offline with considerations for real-time and self-paced control, and a convolutional long-short term memory (C-LSTM) network based on filter bank common spatial patterns (FBCSP) is proposed. In addition, the effects of several methods of data augmentation on different classifiers are explored.The results of this study show that the proposed network achieves adequate results in distinguishing between different control classes, but both considered deep learning models are still less reliable than a Riemannian MDM classifier. In addition, controlled skewing of the data and the explored data augmentation methods improved the average overall accuracy of the classifiers by 14.0% and 5.3%, respectively.MAIN RESULTSThe results of this study show that the proposed network achieves adequate results in distinguishing between different control classes, but both considered deep learning models are still less reliable than a Riemannian MDM classifier. In addition, controlled skewing of the data and the explored data augmentation methods improved the average overall accuracy of the classifiers by 14.0% and 5.3%, respectively.This manuscript is among the first to attempt combining convolutional and recurrent neural network layers for the purpose of MI classification, and is also one of the first to provide an in-depth comparison of various data augmentation methods for MI classification. In addition, all of these methods are applied on smaller windows of data and with consideration to ambient data, which provides a more realistic test bed for real-time and self-paced control.SIGNIFICANCEThis manuscript is among the first to attempt combining convolutional and recurrent neural network layers for the purpose of MI classification, and is also one of the first to provide an in-depth comparison of various data augmentation methods for MI classification. In addition, all of these methods are applied on smaller windows of data and with consideration to ambient data, which provides a more realistic test bed for real-time and self-paced control. |
| Author | Freer, Daniel Yang, Guang-Zhong |
| Author_xml | – sequence: 1 givenname: Daniel orcidid: 0000-0002-4106-4719 surname: Freer fullname: Freer, Daniel email: d.freer15@imperial.ac.uk organization: Author to whom any correspondence should be addressed – sequence: 2 givenname: Guang-Zhong surname: Yang fullname: Yang, Guang-Zhong organization: Shanghai Jiao Tong University Institute of Medical Robotics, Minhang, Shanghai, People's Republic of China |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/31726440$$D View this record in MEDLINE/PubMed |
| BookMark | eNptkb1PwzAQxS1URD9gZ0LZYCD0LrEdd0Tlq1IRA90tJ7FLqiQOcSLU_x5XKZ2Y3un00-m9e1Myqm2tCblGeEAQYo4JxTBiLJqrlCUZnJHJaTU6zRzGZOrcDiDGZAEXZOw14pTChKyeVKcC1W8rXXeqK2wdGNsGTpcmbFSm86CynV8Uldrqdh9kpXKuMEU2sD9F9xUsw_Xn5v2SnBtVOn111BnZvDxvlm_h-uN1tXxchxlF6EK6UGmq8hxTjoYxLhQ3uchAx6gS74qbiItMcKopYxoAAbWPKiKmozzl8YzcDWeb1n732nWyKlymy1LV2vZORjEyEIsEFx69OaJ9WulcNq1P0e7lX3oP3A5AYRu5s31be-NyV2uJiUQJyIGibHLjyft_SAR5qEEe_iwPP5dDDfEvILl3yA |
| CODEN | JNEIEZ |
| CitedBy_id | crossref_primary_10_1109_ACCESS_2020_3044732 crossref_primary_10_1007_s00521_023_08927_w crossref_primary_10_1109_JSEN_2021_3115405 crossref_primary_10_1007_s00521_021_06352_5 crossref_primary_10_1016_j_aei_2024_102434 crossref_primary_10_3390_app12052598 crossref_primary_10_1109_TNSRE_2021_3087506 crossref_primary_10_1016_j_asoc_2023_111129 crossref_primary_10_1007_s00500_023_08837_y crossref_primary_10_1109_JBHI_2023_3337072 crossref_primary_10_1186_s13634_024_01188_2 crossref_primary_10_1109_ACCESS_2024_3351204 crossref_primary_10_1109_LRA_2021_3056355 crossref_primary_10_1016_j_eswa_2025_128678 crossref_primary_10_1109_ACCESS_2020_3011140 crossref_primary_10_1109_TIM_2021_3126832 crossref_primary_10_1109_TIM_2024_3398103 crossref_primary_10_1109_JSEN_2024_3445971 crossref_primary_10_3389_fninf_2025_1521805 crossref_primary_10_1109_TIM_2022_3217515 crossref_primary_10_1007_s11042_023_15900_1 crossref_primary_10_1088_1741_2552_ac1ed0 crossref_primary_10_1016_j_eswa_2024_125585 crossref_primary_10_1109_TNSRE_2021_3099908 crossref_primary_10_1109_TNSRE_2022_3207494 crossref_primary_10_1016_j_bspc_2023_105063 crossref_primary_10_3389_fnhum_2023_1126938 crossref_primary_10_1109_JBHI_2024_3386565 crossref_primary_10_3389_fnhum_2025_1545726 crossref_primary_10_3390_s21155105 crossref_primary_10_1016_j_jneumeth_2020_108885 crossref_primary_10_1109_JSEN_2024_3353146 crossref_primary_10_1109_TNSRE_2025_3527629 crossref_primary_10_3389_fnhum_2021_765525 crossref_primary_10_3390_s20164485 crossref_primary_10_3389_fnhum_2024_1421922 crossref_primary_10_1016_j_artmed_2023_102738 crossref_primary_10_1007_s00521_023_08944_9 crossref_primary_10_1109_TNSRE_2023_3322275 crossref_primary_10_1109_JBHI_2022_3185587 crossref_primary_10_1016_j_neunet_2024_106351 crossref_primary_10_1088_1741_2552_ab6cb9 crossref_primary_10_1109_TNSRE_2022_3167262 crossref_primary_10_3390_bioengineering12050495 crossref_primary_10_1016_j_neucom_2025_131561 crossref_primary_10_1007_s11571_021_09676_z crossref_primary_10_1016_j_bspc_2024_106206 crossref_primary_10_1016_j_bspc_2025_107756 crossref_primary_10_1093_pnasnexus_pgae449 crossref_primary_10_1038_s41598_024_71118_7 crossref_primary_10_1016_j_jneumeth_2022_109593 crossref_primary_10_1016_j_neunet_2025_107516 crossref_primary_10_3390_s25103178 crossref_primary_10_1109_ACCESS_2023_3263489 crossref_primary_10_3389_fnhum_2021_643386 crossref_primary_10_1007_s11432_022_3548_2 crossref_primary_10_1109_JBHI_2024_3496757 crossref_primary_10_1007_s13198_024_02475_9 crossref_primary_10_1007_s13246_023_01316_6 crossref_primary_10_1088_1741_2552_ac4430 crossref_primary_10_1109_TCYB_2021_3052813 crossref_primary_10_1177_15500594221148285 |
| ContentType | Journal Article |
| Copyright | 2020 IOP Publishing Ltd |
| Copyright_xml | – notice: 2020 IOP Publishing Ltd |
| DBID | O3W TSCCA CGR CUY CVF ECM EIF NPM 7X8 |
| DOI | 10.1088/1741-2552/ab57c0 |
| DatabaseName | Institute of Physics Open Access Journal Titles IOPscience (Open Access) Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed MEDLINE - Academic |
| DatabaseTitle | MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) MEDLINE - Academic |
| DatabaseTitleList | MEDLINE MEDLINE - Academic |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: O3W name: Institute of Physics Open Access Journal Titles url: http://iopscience.iop.org/ sourceTypes: Enrichment Source Publisher – sequence: 3 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Anatomy & Physiology |
| DocumentTitleAlternate | Data augmentation for self-paced motor imagery classification with C-LSTM |
| EISSN | 1741-2552 |
| ExternalDocumentID | 31726440 jneab57c0 |
| Genre | Research Support, Non-U.S. Gov't Journal Article |
| GrantInformation_xml | – fundername: Engineering and Physical Sciences Research Council grantid: EP/R026092/1 funderid: https://doi.org/10.13039/501100000266 |
| GroupedDBID | --- 1JI 4.4 53G 5B3 5GY 5VS 5ZH 7.M 7.Q AAGCD AAJIO AAJKP AATNI ABHWH ABJNI ABQJV ABVAM ACAFW ACGFS ACHIP AEFHF AENEX AFYNE AKPSB ALMA_UNASSIGNED_HOLDINGS AOAED ASPBG ATQHT AVWKF AZFZN CEBXE CJUJL CRLBU CS3 DU5 EBS EDWGO EMSAF EPQRW EQZZN F5P HAK IHE IJHAN IOP IZVLO KOT LAP M45 N5L N9A O3W P2P PJBAE RIN RO9 ROL RPA SY9 TSCCA W28 XPP ADEQX CGR CUY CVF ECM EIF NPM 7X8 AEINN |
| ID | FETCH-LOGICAL-c410t-49abbadd1b61f5568a6fd8c0e31a77266f268c864e455e00101e088825e2db63 |
| IEDL.DBID | O3W |
| ISICitedReferencesCount | 72 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000525503200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1741-2560 1741-2552 |
| IngestDate | Sun Aug 24 04:08:25 EDT 2025 Mon Jul 21 06:05:29 EDT 2025 Thu Jan 07 14:56:33 EST 2021 Wed Aug 21 03:33:33 EDT 2024 |
| IsDoiOpenAccess | true |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Language | English |
| License | Content from this work may be used under the terms of the Creative Commons Attribution 4.0 licence. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c410t-49abbadd1b61f5568a6fd8c0e31a77266f268c864e455e00101e088825e2db63 |
| Notes | JNE-103096.R2 ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| ORCID | 0000-0002-4106-4719 |
| OpenAccessLink | https://iopscience.iop.org/article/10.1088/1741-2552/ab57c0 |
| PMID | 31726440 |
| PQID | 2315089719 |
| PQPubID | 23479 |
| PageCount | 15 |
| ParticipantIDs | pubmed_primary_31726440 iop_journals_10_1088_1741_2552_ab57c0 proquest_miscellaneous_2315089719 |
| PublicationCentury | 2000 |
| PublicationDate | 2020-01-31 |
| PublicationDateYYYYMMDD | 2020-01-31 |
| PublicationDate_xml | – month: 01 year: 2020 text: 2020-01-31 day: 31 |
| PublicationDecade | 2020 |
| PublicationPlace | England |
| PublicationPlace_xml | – name: England |
| PublicationTitle | Journal of neural engineering |
| PublicationTitleAbbrev | JNE |
| PublicationTitleAlternate | J. Neural Eng |
| PublicationYear | 2020 |
| Publisher | IOP Publishing |
| Publisher_xml | – name: IOP Publishing |
| SSID | ssj0031790 |
| Score | 2.5109403 |
| Snippet | Objective. Brain-computer interfaces (BCI) are becoming important tools for assistive technology, particularly through the use of motor imagery (MI) for aiding... Brain-computer interfaces (BCI) are becoming important tools for assistive technology, particularly through the use of motor imagery (MI) for aiding task... |
| SourceID | proquest pubmed iop |
| SourceType | Aggregation Database Index Database Enrichment Source Publisher |
| StartPage | 016041 |
| SubjectTerms | brain-computer interface Brain-Computer Interfaces - classification data augmentation Data Science - classification Data Science - methods deep learning electroencephalography Humans Imagination - physiology Motor Cortex - physiology motor imagery Movement - physiology |
| Title | Data augmentation for self-paced motor imagery classification with C-LSTM |
| URI | https://iopscience.iop.org/article/10.1088/1741-2552/ab57c0 https://www.ncbi.nlm.nih.gov/pubmed/31726440 https://www.proquest.com/docview/2315089719 |
| Volume | 17 |
| WOSCitedRecordID | wos000525503200001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1bS-wwEB68PfhyvHv2eCGC-hZt2jRN8Um8oKCr4KL7FpI0kRW3u7i7B_z3TtoqCAqCL6WUoU0mM5NvksxXgN2Y6STl1lJhhKXcGHSp1DiKxmPRpnwWVYw391dZuy273fx2Co4-amEGwyb0H-BtTRRcq7A5ECcPEUMzikg4PtQmzSzm67OJTEUw8pvk4T0MJ4F6qq6GDNIiavYov3oDziv4se8xZjXXnC_8qpWL8KeBmOS4Fl2CKVcuw8pxiel1_5Xsk-rQZ7WavgKXp3qsiZ489psapJIgiiUj9-wpZtOuIDiU-KDXD1wXr8QGsB1OF9WyYRWXnNCru871KnTOzzonF7T5uwK1nEVjynNtDEY3ZgTzgYdMC19IG7mEaYTcQvhYSCsFdzxNXcVF57BjmFG6uDAiWYOZclC6v0CYMDmXnnkdWy6c1IgCpfEFF4XXNpEt2EMtqcY5Rqra95ZSBRWpoCJVq6gF5JPcU-lQRjEVWPBQdFj4Fuy8j5FCPwibG7p0g8lIIU5FrJlnLG_Bej14algTdii0hYD7on8_bMgGzMchrY4YTlGbMDN-mbgtmLP_x73RyzZMZ12J1_bt9XZlcm9ebtIb |
| linkProvider | IOP Publishing |
| linkToPdf | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV3fT9swED6xgqa9jF9jBAZ40sabaZw4rvOIKBWI0lVatfXNsh176kTTiraT-O85J-nDpCEh7S2KLolzd7a_s33fAXxJmE4zbi0VRljKjcEulRlH0Xks-pTvxBXjzY9-ZzCQ43E-bOqcVrkws3kz9F_gZU0UXKuwORAn24ihGUUknLS1yTo2bs8L_wY2szRLQ-2Gb-nP9VCcBvqpOiMyPCHiZp_yX2_BuQU_-DLOrOab3vZ_t3QH3jdQk1zW4ruw4co92L8sMcyePpFzUh3-rFbV9-G2q5ea6NWvaZOLVBJEs2ThHjzFqNoVBE2KNybTwHnxRGwA3eGUUS0bVnPJFe1_H91_gFHvenR1Q5sqC9RyFi8pz7UxOMoxI5gPfGRa-ELa2KVMI_QWwidCWim441nmKk46hz-HkaVLCiPSA2iVs9IdAmHC5Fx65nViuXBSIxqUxhdcFF7bVEbwFTWlmk6yUNX-t5QqqEkFNalaTRGQv-R-lw5lFFOBDQ9FUYsRfF7bSWF_CJscunSz1UIhXkXMmXdYHsHH2oBqXhN3KPSHgP_io1c25AzeDrs91b8d3B3DuyRE2jHDWesTtJaPK3cCW_bPcrJ4PK287hlVb9WG |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Data+augmentation+for+self-paced+motor+imagery+classification+with+C-LSTM&rft.jtitle=Journal+of+neural+engineering&rft.au=Freer%2C+Daniel&rft.au=Yang%2C+Guang-Zhong&rft.date=2020-01-31&rft.pub=IOP+Publishing&rft.issn=1741-2560&rft.eissn=1741-2552&rft.volume=17&rft.issue=1&rft_id=info:doi/10.1088%2F1741-2552%2Fab57c0&rft.externalDocID=jneab57c0 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1741-2560&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1741-2560&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1741-2560&client=summon |