In Silico Mapping of Visual Categorical Selectivity Across the Whole Brain
A fine-grained account of functional selectivity in the cortex is essential for understanding how visual information is processed and represented in the brain. Classical studies using designed experiments have identified multiple category-selective regions; however, these approaches rely on preconce...
Gespeichert in:
| Veröffentlicht in: | ArXiv.org |
|---|---|
| Hauptverfasser: | , , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
United States
24.10.2025
|
| ISSN: | 2331-8422, 2331-8422 |
| Online-Zugang: | Volltext |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | A fine-grained account of functional selectivity in the cortex is essential for understanding how visual information is processed and represented in the brain. Classical studies using designed experiments have identified multiple category-selective regions; however, these approaches rely on preconceived hypotheses about categories. Subsequent data-driven discovery methods have sought to address this limitation but are often limited by simple, typically linear encoding models. We propose an in silico approach for data-driven discovery of novel category-selectivity hypotheses based on an encoder-decoder transformer model. The architecture incorporates a brain-region to image-feature cross-attention mechanism, enabling nonlinear mappings between high-dimensional deep network features and semantic patterns encoded in the brain activity. We further introduce a method to characterize the selectivity of individual parcels by leveraging diffusion-based image generative models and large-scale datasets to synthesize and select images that maximally activate each parcel. Our approach reveals regions with complex, compositional selectivity involving diverse semantic concepts, which we validate in silico both within and across subjects. Using a brain encoder as a "digital twin" offers a powerful, data-driven framework for generating and testing hypotheses about visual selectivity in the human brain-hypotheses that can guide future fMRI experiments. Our code is available at: https://kriegeskorte-lab.github.io/in-silico-mapping/. |
|---|---|
| AbstractList | A fine-grained account of functional selectivity in the cortex is essential for understanding how visual information is processed and represented in the brain. Classical studies using designed experiments have identified multiple category-selective regions; however, these approaches rely on preconceived hypotheses about categories. Subsequent data-driven discovery methods have sought to address this limitation but are often limited by simple, typically linear encoding models. We propose an in silico approach for data-driven discovery of novel category-selectivity hypotheses based on an encoder-decoder transformer model. The architecture incorporates a brain-region to image-feature cross-attention mechanism, enabling nonlinear mappings between high-dimensional deep network features and semantic patterns encoded in the brain activity. We further introduce a method to characterize the selectivity of individual parcels by leveraging diffusion-based image generative models and large-scale datasets to synthesize and select images that maximally activate each parcel. Our approach reveals regions with complex, compositional selectivity involving diverse semantic concepts, which we validate in silico both within and across subjects. Using a brain encoder as a "digital twin" offers a powerful, data-driven framework for generating and testing hypotheses about visual selectivity in the human brain - hypotheses that can guide future fMRI experiments. Our code is available at: https://kriegeskorte-lab.github.io/in-silico-mapping/ .A fine-grained account of functional selectivity in the cortex is essential for understanding how visual information is processed and represented in the brain. Classical studies using designed experiments have identified multiple category-selective regions; however, these approaches rely on preconceived hypotheses about categories. Subsequent data-driven discovery methods have sought to address this limitation but are often limited by simple, typically linear encoding models. We propose an in silico approach for data-driven discovery of novel category-selectivity hypotheses based on an encoder-decoder transformer model. The architecture incorporates a brain-region to image-feature cross-attention mechanism, enabling nonlinear mappings between high-dimensional deep network features and semantic patterns encoded in the brain activity. We further introduce a method to characterize the selectivity of individual parcels by leveraging diffusion-based image generative models and large-scale datasets to synthesize and select images that maximally activate each parcel. Our approach reveals regions with complex, compositional selectivity involving diverse semantic concepts, which we validate in silico both within and across subjects. Using a brain encoder as a "digital twin" offers a powerful, data-driven framework for generating and testing hypotheses about visual selectivity in the human brain - hypotheses that can guide future fMRI experiments. Our code is available at: https://kriegeskorte-lab.github.io/in-silico-mapping/ . A fine-grained account of functional selectivity in the cortex is essential for understanding how visual information is processed and represented in the brain. Classical studies using designed experiments have identified multiple category-selective regions; however, these approaches rely on preconceived hypotheses about categories. Subsequent data-driven discovery methods have sought to address this limitation but are often limited by simple, typically linear encoding models. We propose an in silico approach for data-driven discovery of novel category-selectivity hypotheses based on an encoder-decoder transformer model. The architecture incorporates a brain-region to image-feature cross-attention mechanism, enabling nonlinear mappings between high-dimensional deep network features and semantic patterns encoded in the brain activity. We further introduce a method to characterize the selectivity of individual parcels by leveraging diffusion-based image generative models and large-scale datasets to synthesize and select images that maximally activate each parcel. Our approach reveals regions with complex, compositional selectivity involving diverse semantic concepts, which we validate in silico both within and across subjects. Using a brain encoder as a "digital twin" offers a powerful, data-driven framework for generating and testing hypotheses about visual selectivity in the human brain-hypotheses that can guide future fMRI experiments. Our code is available at: https://kriegeskorte-lab.github.io/in-silico-mapping/. |
| Author | Luo, Andrew Kriegeskorte, Nikolaus Guo, Wenxuan Hwang, Ethan Adeli, Hossein |
| Author_xml | – sequence: 1 givenname: Ethan surname: Hwang fullname: Hwang, Ethan organization: Zuckerman Mind Brain Behavior Institute, Columbia University – sequence: 2 givenname: Hossein surname: Adeli fullname: Adeli, Hossein organization: Zuckerman Mind Brain Behavior Institute, Columbia University – sequence: 3 givenname: Wenxuan surname: Guo fullname: Guo, Wenxuan organization: Zuckerman Mind Brain Behavior Institute, Columbia University – sequence: 4 givenname: Andrew surname: Luo fullname: Luo, Andrew organization: University of Hong Kong – sequence: 5 givenname: Nikolaus surname: Kriegeskorte fullname: Kriegeskorte, Nikolaus organization: Zuckerman Mind Brain Behavior Institute, Columbia University |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/41281227$$D View this record in MEDLINE/PubMed |
| BookMark | eNpNj11LwzAYhYNM3Jz7C5JLbwrtmyZpL2fxYzLxYkMvS5q-3SJpUptW2L-34ASvznM4hwPnmsycd3hBFsBYEmUpwOwfz8kqhM84jkFI4JxdkXmaQJYAyAV52Ti6M9ZoT19V1xl3oL6h7yaMytJCDXjwvdET79CiHsy3GU50rXsfAh2OSD-O3iK975VxN-SyUTbg6qxLsn982BfP0fbtaVOst1HHhYwaUWVKSQlM1DJBlct0cjxniBVgnchYs8lIWee8FkpowLTSuWC5gHhK2ZLc_c52vf8aMQxla4JGa5VDP4aSgUzzNBaCT9Xbc3WsWqzLrjet6k_l3332A9TzWE4 |
| ContentType | Journal Article |
| DBID | NPM 7X8 |
| DatabaseName | PubMed MEDLINE - Academic |
| DatabaseTitle | PubMed MEDLINE - Academic |
| DatabaseTitleList | MEDLINE - Academic PubMed |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Physics |
| EISSN | 2331-8422 |
| ExternalDocumentID | 41281227 |
| Genre | Journal Article Preprint |
| GrantInformation_xml | – fundername: NINDS NIH HHS grantid: RF1 NS128897 – fundername: NINDS NIH HHS grantid: R01 NS128897 |
| GroupedDBID | ABJCF AFKRA ALMA_UNASSIGNED_HOLDINGS BENPR BGLVJ CCPQU FRJ HCIFZ M7S M~E NPM PHGZM PHGZT PIMPY PQGLB PTHSS 7X8 |
| ID | FETCH-LOGICAL-p567-f6b8aa77236d71ea974a77593eeb2ed170c393e77d95d6a6c2e4bc96396201703 |
| ISSN | 2331-8422 |
| IngestDate | Mon Nov 24 17:30:57 EST 2025 Tue Dec 02 01:41:31 EST 2025 |
| IsPeerReviewed | false |
| IsScholarly | false |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-p567-f6b8aa77236d71ea974a77593eeb2ed170c393e77d95d6a6c2e4bc96396201703 |
| Notes | ObjectType-Working Paper/Pre-Print-3 ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| PMID | 41281227 |
| PQID | 3274940665 |
| PQPubID | 23479 |
| ParticipantIDs | proquest_miscellaneous_3274940665 pubmed_primary_41281227 |
| PublicationCentury | 2000 |
| PublicationDate | 2025-Oct-24 20251024 |
| PublicationDateYYYYMMDD | 2025-10-24 |
| PublicationDate_xml | – month: 10 year: 2025 text: 2025-Oct-24 day: 24 |
| PublicationDecade | 2020 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | ArXiv.org |
| PublicationTitleAlternate | ArXiv |
| PublicationYear | 2025 |
| SSID | ssj0002672553 |
| Score | 1.9259036 |
| SecondaryResourceType | preprint |
| Snippet | A fine-grained account of functional selectivity in the cortex is essential for understanding how visual information is processed and represented in the brain.... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| Title | In Silico Mapping of Visual Categorical Selectivity Across the Whole Brain |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/41281227 https://www.proquest.com/docview/3274940665 |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVHPJ databaseName: ROAD: Directory of Open Access Scholarly Resources customDbUrl: eissn: 2331-8422 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0002672553 issn: 2331-8422 databaseCode: M~E dateStart: 19910101 isFulltext: true titleUrlDefault: https://road.issn.org providerName: ISSN International Centre – providerCode: PRVPQU databaseName: Engineering Database customDbUrl: eissn: 2331-8422 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0002672553 issn: 2331-8422 databaseCode: M7S dateStart: 19880101 isFulltext: true titleUrlDefault: http://search.proquest.com providerName: ProQuest – providerCode: PRVPQU databaseName: ProQuest Central customDbUrl: eissn: 2331-8422 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0002672553 issn: 2331-8422 databaseCode: BENPR dateStart: 19880101 isFulltext: true titleUrlDefault: https://www.proquest.com/central providerName: ProQuest – providerCode: PRVPQU databaseName: Publicly Available Content Database customDbUrl: eissn: 2331-8422 dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0002672553 issn: 2331-8422 databaseCode: PIMPY dateStart: 19880101 isFulltext: true titleUrlDefault: http://search.proquest.com/publiccontent providerName: ProQuest |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtZ3Pb9MwFMefaAFpF8TvDVhlJMQFBS22EyfHMXVbUVcqFlg4RY7jSpFQGpZ19M_n-QdZDiCNA5coflJ8yCfxr_fjC_AGpwwtuSyDMF4lAedCBaUQkSkhagSQBI-1TRSei8UiyfN06eUtOysnIJom2W7T9r-iRhvCNqmz_4C77xQNeI_Q8YrY8Xor8DP8YevvCPjdmWxbH9T8te5MnsiRqQvhq4KcWwUcpx1xaCdLuwi9MIK5iFz6ktx9hdq8vjYxD_138NOfNE_N4fuN21-7fOtT7E_Xvf1kY49kL3Sz3QyigJx1EFTpzx-orVvq0p7dMEUZC4OEu-zi9_pPtmGR68Wn4vjLfF5k0zx72_4IjP6X8ZN7MZQRjNgBH8PdD9PF8nN_WkZjgXsfZpUCfdd_3xLYpUH2EB74NT05dCwewR3dPIb7NrZWdU_g46whjgjxRMh6RRwRMiBCBkSII0KQCLFEiCXyFLLjaXZ0GngJi6CNcAZaxWUiJW5gWFyJUEv89rEVpUzrkuoqFAeKYUOIKo2qWMaKal4qHBPTmJrCRuwZjJt1o3eB4B8VKkZFYlyhJl1ZKKXKikZlhavustqD179fR4EjhHH7yEavN12BD_GUGxfbHjx376loXSmTghtHKqXixS2efgk7N_xfwfjqcqP34Z66vqq7ywmMRJ5MPLeJCaE9x9Zydrb89gubqkGI |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=In+Silico+Mapping+of+Visual+Categorical+Selectivity+Across+the+Whole+Brain&rft.jtitle=ArXiv.org&rft.au=Hwang%2C+Ethan&rft.au=Adeli%2C+Hossein&rft.au=Guo%2C+Wenxuan&rft.au=Luo%2C+Andrew&rft.date=2025-10-24&rft.issn=2331-8422&rft.eissn=2331-8422&rft.externalDBID=NO_FULL_TEXT |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2331-8422&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2331-8422&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2331-8422&client=summon |