An LSTM-based adversarial variational autoencoder framework for self-supervised neural decoding of behavioral choices

.This paper presents data-driven solutions to address two challenges in the problem of linking neural data and behavior: (1) unsupervised analysis of behavioral data and automatic label generation from behavioral observations, and (2) extraction of subject-invariant features for the development of g...

Celý popis

Uložené v:

Podrobná bibliografia
Vydané v:	Journal of neural engineering Ročník 21; číslo 3
Hlavní autori:	Salsabilian, Shiva, Lee, Christian, Margolis, David, Najafizadeh, Laleh
Médium:	Journal Article
Jazyk:	English
Vydavateľské údaje:	England IOP Publishing 01.06.2024
Predmet:	adversarial variational autoencoder Animals behavior Choice Behavior - physiology LSTM Mice Mice, Transgenic neural decoding Neural Networks, Computer self-supervised learning subject variability Supervised Machine Learning Unsupervised Machine Learning neural decoding self-supervised learning adversarial variational autoencoder LSTM behavior subject variability
ISSN:	1741-2560, 1741-2552, 1741-2552
On-line prístup:	Získať plný text
Tagy:	Pridať tag Žiadne tagy, Buďte prvý, kto otaguje tento záznam!

Abstract	.This paper presents data-driven solutions to address two challenges in the problem of linking neural data and behavior: (1) unsupervised analysis of behavioral data and automatic label generation from behavioral observations, and (2) extraction of subject-invariant features for the development of generalized neural decoding models. . For behavioral analysis and label generation, an unsupervised method, which employs an autoencoder to transform behavioral data into a cluster-friendly feature space is presented. The model iteratively refines the assigned clusters with soft clustering assignment loss, and gradually improves the learned feature representations. To address subject variability in decoding neural activity, adversarial learning in combination with a long short-term memory-based adversarial variational autoencoder (LSTM-AVAE) model is employed. By using an adversary network to constrain the latent representations, the model captures shared information among subjects' neural activity, making it proper for cross-subject transfer learning. . The proposed approach is evaluated using cortical recordings of Thy1-GCaMP6s transgenic mice obtained via widefield calcium imaging during a motivational licking behavioral experiment. The results show that the proposed model achieves an accuracy of 89.7% in cross-subject neural decoding, outperforming other well-known autoencoder-based feature learning models. These findings suggest that incorporating an adversary network eliminates subject dependency in representations, leading to improved cross-subject transfer learning performance, while also demonstrating the effectiveness of LSTM-based models in capturing the temporal dependencies within neural data. . Results demonstrate the feasibility of the proposed framework in unsupervised clustering and label generation of behavioral data, as well as achieving high accuracy in cross-subject neural decoding, indicating its potentials for relating neural activity to behavior.
AbstractList	Objective.This paper presents data-driven solutions to address two challenges in the problem of linking neural data and behavior: (1) unsupervised analysis of behavioral data and automatic label generation from behavioral observations, and (2) extraction of subject-invariant features for the development of generalized neural decoding models.Approach. For behavioral analysis and label generation, an unsupervised method, which employs an autoencoder to transform behavioral data into a cluster-friendly feature space is presented. The model iteratively refines the assigned clusters with soft clustering assignment loss, and gradually improves the learned feature representations. To address subject variability in decoding neural activity, adversarial learning in combination with a long short-term memory-based adversarial variational autoencoder (LSTM-AVAE) model is employed. By using an adversary network to constrain the latent representations, the model captures shared information among subjects' neural activity, making it proper for cross-subject transfer learning.Main results. The proposed approach is evaluated using cortical recordings of Thy1-GCaMP6s transgenic mice obtained via widefield calcium imaging during a motivational licking behavioral experiment. The results show that the proposed model achieves an accuracy of 89.7% in cross-subject neural decoding, outperforming other well-known autoencoder-based feature learning models. These findings suggest that incorporating an adversary network eliminates subject dependency in representations, leading to improved cross-subject transfer learning performance, while also demonstrating the effectiveness of LSTM-based models in capturing the temporal dependencies within neural data.Significance. Results demonstrate the feasibility of the proposed framework in unsupervised clustering and label generation of behavioral data, as well as achieving high accuracy in cross-subject neural decoding, indicating its potentials for relating neural activity to behavior.Objective.This paper presents data-driven solutions to address two challenges in the problem of linking neural data and behavior: (1) unsupervised analysis of behavioral data and automatic label generation from behavioral observations, and (2) extraction of subject-invariant features for the development of generalized neural decoding models.Approach. For behavioral analysis and label generation, an unsupervised method, which employs an autoencoder to transform behavioral data into a cluster-friendly feature space is presented. The model iteratively refines the assigned clusters with soft clustering assignment loss, and gradually improves the learned feature representations. To address subject variability in decoding neural activity, adversarial learning in combination with a long short-term memory-based adversarial variational autoencoder (LSTM-AVAE) model is employed. By using an adversary network to constrain the latent representations, the model captures shared information among subjects' neural activity, making it proper for cross-subject transfer learning.Main results. The proposed approach is evaluated using cortical recordings of Thy1-GCaMP6s transgenic mice obtained via widefield calcium imaging during a motivational licking behavioral experiment. The results show that the proposed model achieves an accuracy of 89.7% in cross-subject neural decoding, outperforming other well-known autoencoder-based feature learning models. These findings suggest that incorporating an adversary network eliminates subject dependency in representations, leading to improved cross-subject transfer learning performance, while also demonstrating the effectiveness of LSTM-based models in capturing the temporal dependencies within neural data.Significance. Results demonstrate the feasibility of the proposed framework in unsupervised clustering and label generation of behavioral data, as well as achieving high accuracy in cross-subject neural decoding, indicating its potentials for relating neural activity to behavior. .This paper presents data-driven solutions to address two challenges in the problem of linking neural data and behavior: (1) unsupervised analysis of behavioral data and automatic label generation from behavioral observations, and (2) extraction of subject-invariant features for the development of generalized neural decoding models. . For behavioral analysis and label generation, an unsupervised method, which employs an autoencoder to transform behavioral data into a cluster-friendly feature space is presented. The model iteratively refines the assigned clusters with soft clustering assignment loss, and gradually improves the learned feature representations. To address subject variability in decoding neural activity, adversarial learning in combination with a long short-term memory-based adversarial variational autoencoder (LSTM-AVAE) model is employed. By using an adversary network to constrain the latent representations, the model captures shared information among subjects' neural activity, making it proper for cross-subject transfer learning. . The proposed approach is evaluated using cortical recordings of Thy1-GCaMP6s transgenic mice obtained via widefield calcium imaging during a motivational licking behavioral experiment. The results show that the proposed model achieves an accuracy of 89.7% in cross-subject neural decoding, outperforming other well-known autoencoder-based feature learning models. These findings suggest that incorporating an adversary network eliminates subject dependency in representations, leading to improved cross-subject transfer learning performance, while also demonstrating the effectiveness of LSTM-based models in capturing the temporal dependencies within neural data. . Results demonstrate the feasibility of the proposed framework in unsupervised clustering and label generation of behavioral data, as well as achieving high accuracy in cross-subject neural decoding, indicating its potentials for relating neural activity to behavior.
Author	Salsabilian, Shiva Margolis, David Najafizadeh, Laleh Lee, Christian
Author_xml	– sequence: 1 givenname: Shiva surname: Salsabilian fullname: Salsabilian, Shiva organization: Rutgers University Integrated Systems and NeuroImaging Laboratory, Department of Electrical and Computer Engineering, Piscataway, NJ 08854, United States of America – sequence: 2 givenname: Christian orcidid: 0000-0003-2475-9567 surname: Lee fullname: Lee, Christian organization: Rutgers University Department of Cell Biology and Neuroscience, Piscataway, NJ, United States of America – sequence: 3 givenname: David orcidid: 0000-0002-2678-4216 surname: Margolis fullname: Margolis, David organization: Rutgers University Department of Cell Biology and Neuroscience, Piscataway, NJ, United States of America – sequence: 4 givenname: Laleh orcidid: 0000-0002-6658-4112 surname: Najafizadeh fullname: Najafizadeh, Laleh organization: Rutgers University Integrated Systems and NeuroImaging Laboratory, Department of Electrical and Computer Engineering, Piscataway, NJ 08854, United States of America
BackLink	https://www.ncbi.nlm.nih.gov/pubmed/38621379$$D View this record in MEDLINE/PubMed
BookMark	eNptkb1PwzAUxC0E4ntnQtlgIGDHteOMCPElFTFQZuslfgaX1A52U8R_T6IWJqY7Pf3eDXcHZNsHj4ScMHrJqFJXrJywvBCiuALDseZbZP_vtP3nJd0jBynNKeWsrOgu2eNKFoyX1T7pr302fZk95TUkNBmYFcYE0UGbrUZZuuAHD_0yoG-CwZjZCAv8CvEjsyFmCVubp77DuHJjgsc-Dg8GB9j5tyzYrMZ3WLkwnpv34BpMR2THQpvweKOH5PXudnbzkE-f7x9vrqd5M2F0mTfC8kZZlAa4sUaKGpUtigkUqrRUcC5YVWMpyrIqaEOVBahkLQzIyggFgh-S83VuF8Nnj2mpFy412LbgMfRJc8orRaWc8AE93aB9vUCju-gWEL_1b1UDcLEGXOj0PPRx6CVpRvW4hB6r1mPter3EgJ_9g8896oJprimXdIA7Y_kPduqKpQ
CODEN	JNEOBH
CitedBy_id	crossref_primary_10_1109_ACCESS_2025_3582805
ContentType	Journal Article
Copyright	2024 The Author(s). Published by IOP Publishing Ltd Creative Commons Attribution license.
Copyright_xml	– notice: 2024 The Author(s). Published by IOP Publishing Ltd – notice: Creative Commons Attribution license.
DBID	O3W TSCCA CGR CUY CVF ECM EIF NPM 7X8
DOI	10.1088/1741-2552/ad3eb3
DatabaseName	Institute of Physics Open Access Journal Titles IOPscience (Open Access) Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed MEDLINE - Academic
DatabaseTitle	MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) MEDLINE - Academic
DatabaseTitleList	MEDLINE - Academic MEDLINE
Database_xml	– sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: O3W name: Institute of Physics Open Access Journal Titles url: http://iopscience.iop.org/ sourceTypes: Enrichment Source Publisher – sequence: 3 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database
DeliveryMethod	fulltext_linktorsrc
Discipline	Anatomy & Physiology
EISSN	1741-2552
ExternalDocumentID	38621379 jnead3eb3
Genre	Research Support, U.S. Gov't, Non-P.H.S Journal Article
GrantInformation_xml	– fundername: Division of Chemical, Bioengineering, Environmental, and Transport Systems grantid: 1605646 funderid: http://dx.doi.org/10.13039/100000146
GroupedDBID	--- 1JI 4.4 53G 5B3 5GY 5VS 5ZH 7.M 7.Q AAGCD AAJIO AAJKP AATNI ABHWH ABJNI ABQJV ABVAM ACAFW ACGFS ACHIP AEFHF AENEX AFYNE AKPSB ALMA_UNASSIGNED_HOLDINGS AOAED ASPBG ATQHT AVWKF AZFZN CEBXE CJUJL CRLBU CS3 DU5 EBS EDWGO EMSAF EPQRW EQZZN F5P HAK IHE IJHAN IOP IZVLO KOT LAP N5L N9A O3W P2P PJBAE RIN RO9 ROL RPA SY9 TSCCA W28 XPP ADEQX CGR CUY CVF ECM EIF NPM 7X8 AEINN
ID	FETCH-LOGICAL-c410t-c5f3c8fe6da3dfd65be8f224a287f0533519be7577920c08faa96b5da69d58a53
IEDL.DBID	O3W
ISICitedReferencesCount	1
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001268129000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	1741-2560 1741-2552
IngestDate	Thu Oct 02 07:01:09 EDT 2025 Thu May 08 05:31:01 EDT 2025 Sun Aug 18 17:40:26 EDT 2024 Tue Aug 20 22:17:05 EDT 2024
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	3
Keywords	neural decoding self-supervised learning adversarial variational autoencoder LSTM behavior subject variability
Language	English
License	Original content from this work may be used under the terms of the Creative Commons Attribution 4.0 license. Any further distribution of this work must maintain attribution to the author(s) and the title of the work, journal citation and DOI. Creative Commons Attribution license.
LinkModel	DirectLink
MergedId	FETCHMERGED-LOGICAL-c410t-c5f3c8fe6da3dfd65be8f224a287f0533519be7577920c08faa96b5da69d58a53
Notes	JNE-106846.R1 ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23
ORCID	0000-0003-2475-9567 0000-0002-6658-4112 0000-0002-2678-4216
OpenAccessLink	https://iopscience.iop.org/article/10.1088/1741-2552/ad3eb3
PMID	38621379
PQID	3039806643
PQPubID	23479
PageCount	15
ParticipantIDs	pubmed_primary_38621379 iop_journals_10_1088_1741_2552_ad3eb3 proquest_miscellaneous_3039806643
PublicationCentury	2000
PublicationDate	2024-06-01
PublicationDateYYYYMMDD	2024-06-01
PublicationDate_xml	– month: 06 year: 2024 text: 2024-06-01 day: 01
PublicationDecade	2020
PublicationPlace	England
PublicationPlace_xml	– name: England
PublicationTitle	Journal of neural engineering
PublicationTitleAbbrev	JNE
PublicationTitleAlternate	J. Neural Eng
PublicationYear	2024
Publisher	IOP Publishing
Publisher_xml	– name: IOP Publishing
SSID	ssj0031790
Score	2.4008627
Snippet	.This paper presents data-driven solutions to address two challenges in the problem of linking neural data and behavior: (1) unsupervised analysis of... Objective.This paper presents data-driven solutions to address two challenges in the problem of linking neural data and behavior: (1) unsupervised analysis of...
SourceID	proquest pubmed iop
SourceType	Aggregation Database Index Database Enrichment Source Publisher
SubjectTerms	adversarial variational autoencoder Animals behavior Choice Behavior - physiology LSTM Mice Mice, Transgenic neural decoding Neural Networks, Computer self-supervised learning subject variability Supervised Machine Learning Unsupervised Machine Learning
Title	An LSTM-based adversarial variational autoencoder framework for self-supervised neural decoding of behavioral choices
URI	https://iopscience.iop.org/article/10.1088/1741-2552/ad3eb3 https://www.ncbi.nlm.nih.gov/pubmed/38621379 https://www.proquest.com/docview/3039806643
Volume	21
WOSCitedRecordID	wos001268129000001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1Lb9QwEB61hQOX8iiP5VEZCbiFJvE6scVphag4tEslitibNfFDAkGy2mwq9d8zE2dBSCAhcUlycCbW2JnvG3tmDPAiSl_E3DlibpySQxCVoZnrTBkkdouOIGGsM3tWL5d6tTIXe_DmZy5Mt55M_2t6TIWCkwqngDh9Qhy6yIgJlyfoJfmC-3BDalXxJP8gP-_MsOTSUykbkltX-bRH-ScJhCv0sb9zzBFrTm__Vy_vwOFEMcUiNb0Le6G9B0eLltzr79filRiDPsfV9CMYFq04-3h5njGaeYF8PHOPPCnFFd_SSqHAYdtxxUsfNiLuwrkE8V3Rh28x64c1mxyWwPUx6QVPXi2jouii-FUJQJCtZcN0Hz6dvrt8-z6bTmLI3LzIt5lTUTodQ-VR-ugr1QQdCfyR_K3I2bzEA5tQq7o2Ze5yHRFN1SiPlfFKo5IP4KDt2vAIhFTV3NUOSz70J2KBpqkNEoD6RjUkewYvSaV2-pN6O26Sa21Zn5b1aZM-ZyB-a_e1DbYsrLQEyTm1Wvs4g-e7AbX00_BOCLahG3pLuG00ka05iXmYRtquU3UPK8nHK2RtHv9jR57ArZKITgofewoH280QnsFNd7X90m-OYb9eabouL86Px_n5A2Sm5Sw
linkProvider	IOP Publishing
linkToPdf	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1Lb9QwELZKi1AvpVBot7yMBNzSTeJ1Yh9XwKqIZalEEb1ZEz8kqjZZbTaV-u87E2eFkEBC4pQcbCca2_N9Y8-DsTdBuCyk1iJzo5AchKgE9EQlUgOyW7AICX2e2Xm5WKiLC3021DntY2Ga5aD6T_A1JgqOIhwc4tQYOXSWIBPOx-AE2oLjpQv32I4UUlDthq_ix0YVC0o_FSMiqUeRDveUfxoFsQU_-Hee2ePN7OF__-k-2xuoJp_G5o_Ylq8fs4NpjWb29S1_x3vnz_5U_YB105rPv51_SQjVHAcq09wCLU5-Q494YsihWzeU-dL5FQ8bty6OvJe3_iokbbck1UMjUJ5M7ODQuiV05E3gvzICcNS5pKCesO-zj-fvT5OhIkNiJ1m6TqwMwqrgCwfCBVfIyquAJADQ7goU1Yt8sPKlLEudpzZVAUAXlXRQaCcVSPGUbddN7Y8YF7KY2NJCTsV_AmSgq1IDAqmrZIVjj9hbFKsZdlRr-stypQzJ1JBMTZTpiPHf2l3W3uSZEQahOcVWKPIRe72ZVIObh25EoPZN1xrEb62QdE1wmMM422YZs3wYgbZeJkp9_I8_8oo9OPswM_NPi8_P2G6O3Cd6lD1n2-tV51-w-_Zm_bNdveyX6B3H2eiX
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=An+LSTM-based+adversarial+variational+autoencoder+framework+for+self-supervised+neural+decoding+of+behavioral+choices&rft.jtitle=Journal+of+neural+engineering&rft.au=Salsabilian%2C+Shiva&rft.au=Lee%2C+Christian&rft.au=Margolis%2C+David&rft.au=Najafizadeh%2C+Laleh&rft.date=2024-06-01&rft.pub=IOP+Publishing&rft.issn=1741-2560&rft.eissn=1741-2552&rft.volume=21&rft.issue=3&rft_id=info:doi/10.1088%2F1741-2552%2Fad3eb3&rft.externalDocID=jnead3eb3
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1741-2560&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1741-2560&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1741-2560&client=summon