Investigating the brain network characteristics of multimodal emotion recognition and its classification applications based on functional connectivity patterns
•The differences in multimodal affective cognitive brain networks were further discovered.•Computer algorithms provide a more sufficient theoretical basis for cognitive task decoding.•Model fine-tuning and feature selection enriched the relevant results and inspired future research.•Different angles...
Uloženo v:
| Vydáno v: | Biomedical signal processing and control Ročník 96; s. 106635 |
|---|---|
| Hlavní autoři: | , , , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
Elsevier Ltd
01.10.2024
|
| Témata: | |
| ISSN: | 1746-8094 |
| On-line přístup: | Získat plný text |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | •The differences in multimodal affective cognitive brain networks were further discovered.•Computer algorithms provide a more sufficient theoretical basis for cognitive task decoding.•Model fine-tuning and feature selection enriched the relevant results and inspired future research.•Different angles results were combined and analyzed by different algorithms to improve the confidence of the results.
Emotion information can be expressed in multiple modal stimuli, and the brain can recognize multi-modal emotions efficiently and accurately. Recent researches have thoroughly analyzed the activity characteristics of relevant brain regions under different modal emotion information based on functional magnetic resonance imaging (fMRI). However, considering the functional integration characteristics of the brain in cognitive activities, further research on the brain network in the process of multi-modal emotion recognition should be carried out, which can reveal the brain’s multi-modal emotion cognition mechanism more comprehensively. In this study, functional connectivity (FC) analysis was performed on the fMRI data from multimodal emotion recognition tasks. The correlation coefficients of brain regions were calculated and statistically analyzed to study the characteristics of FC patterns in multimodal emotion recognition processing. Moreover, the emotional information was decoded with different machine learning classification algorithms based on the FC patterns. The results showed that the modal and valence of emotion can lead to structural similarities and connection strength differences in connection strength of brain region connections in the brain network, and this property can successfully support the decoding of emotional information, and the decoding accuracy is higher than the previous decoding accuracy based on brain region activation patterns. This study explores the cognitive mechanism of multi-modal emotion recognition from a new perspective of brain functional integration, and compensates for the lack of brain signal decoding methods based on FC features and machine learning methods. |
|---|---|
| AbstractList | •The differences in multimodal affective cognitive brain networks were further discovered.•Computer algorithms provide a more sufficient theoretical basis for cognitive task decoding.•Model fine-tuning and feature selection enriched the relevant results and inspired future research.•Different angles results were combined and analyzed by different algorithms to improve the confidence of the results.
Emotion information can be expressed in multiple modal stimuli, and the brain can recognize multi-modal emotions efficiently and accurately. Recent researches have thoroughly analyzed the activity characteristics of relevant brain regions under different modal emotion information based on functional magnetic resonance imaging (fMRI). However, considering the functional integration characteristics of the brain in cognitive activities, further research on the brain network in the process of multi-modal emotion recognition should be carried out, which can reveal the brain’s multi-modal emotion cognition mechanism more comprehensively. In this study, functional connectivity (FC) analysis was performed on the fMRI data from multimodal emotion recognition tasks. The correlation coefficients of brain regions were calculated and statistically analyzed to study the characteristics of FC patterns in multimodal emotion recognition processing. Moreover, the emotional information was decoded with different machine learning classification algorithms based on the FC patterns. The results showed that the modal and valence of emotion can lead to structural similarities and connection strength differences in connection strength of brain region connections in the brain network, and this property can successfully support the decoding of emotional information, and the decoding accuracy is higher than the previous decoding accuracy based on brain region activation patterns. This study explores the cognitive mechanism of multi-modal emotion recognition from a new perspective of brain functional integration, and compensates for the lack of brain signal decoding methods based on FC features and machine learning methods. |
| ArticleNumber | 106635 |
| Author | Gu, Jin Gong, Xinhao Luo, Xiaoqi Su, Chenxu |
| Author_xml | – sequence: 1 givenname: Jin orcidid: 0000-0003-1147-6170 surname: Gu fullname: Gu, Jin email: gujin@swjtu.edu.cn organization: School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, China – sequence: 2 givenname: Xiaoqi surname: Luo fullname: Luo, Xiaoqi organization: School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, China – sequence: 3 givenname: Xinhao surname: Gong fullname: Gong, Xinhao organization: School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, China – sequence: 4 givenname: Chenxu surname: Su fullname: Su, Chenxu organization: School of Computing and Artificial Intelligence, Southwest Jiaotong University, Chengdu, China |
| BookMark | eNp9kE1uAjEMhbOgUoH2Al3lAtDxTOYHqZsK9QcJqZt2HXmcDIQOySgJVJymV20oXXfl5yc_2_ombGSd1YzdQTaHDKr73bwNA83zLBfJqKqiHLEx1KKaNdlCXLNJCLssE00NYsy-V_aoQzQbjMZueNxq3no0llsdv5z_5LRFjxS1N2mKAncd3x_6aPZOYc_13kXjLPea3MaaX41WcRMDpx5DMJ0hvNjD0P_pwFsMWvHkdgdLZyvtImetTs3RxBMfMKabNtywqw77oG__6pR9PD-9L19n67eX1fJxPaO8hDirG6FEQ1AvoMBCQQ6Luso7QlIgqjIHqqlTLRbQEOWkSo2iVaKGhWgBSiymLL_sJe9C8LqTgzd79CcJmTxjlTt5xirPWOUFawo9XEI6fXY02stARlvSyiQgUSpn_ov_ABO9izM |
| Cites_doi | 10.15837/ijccc.2019.4.3433 10.1109/TNSRE.2020.3037326 10.1007/s00406-021-01354-9 10.1007/s11682-021-00567-9 10.1002/hbm.26482 10.1007/s11042-020-10030-4 10.1523/JNEUROSCI.4059-14.2015 10.1109/ICACI52617.2021.9435906 10.1523/JNEUROSCI.2161-10.2010 10.1212/WNL.0000000000201495 10.1016/j.neuron.2019.04.023 10.3390/s21144913 10.1523/JNEUROSCI.0217-11.2011 10.1093/cercor/bhac534 10.1007/s10072-020-04794-8 10.3390/brainsci10100703 10.1016/j.neuroimage.2019.116199 10.1016/j.neuron.2022.05.015 10.3389/fpsyg.2023.1188634 10.1073/pnas.1900390116 10.1093/cercor/bhv086 10.1016/j.neuroscience.2021.06.002 10.1073/pnas.0811168106 10.1007/s11682-019-00135-2 10.1007/s00429-022-02582-y 10.1088/1741-2552/ac49a7 10.1016/j.neuroimage.2017.01.002 10.3389/fnhum.2022.960784 |
| ContentType | Journal Article |
| Copyright | 2024 Elsevier Ltd |
| Copyright_xml | – notice: 2024 Elsevier Ltd |
| DBID | AAYXX CITATION |
| DOI | 10.1016/j.bspc.2024.106635 |
| DatabaseName | CrossRef |
| DatabaseTitle | CrossRef |
| DatabaseTitleList | |
| DeliveryMethod | fulltext_linktorsrc |
| Discipline | Engineering |
| ExternalDocumentID | 10_1016_j_bspc_2024_106635 S1746809424006931 |
| GroupedDBID | --- --K --M .~1 0R~ 1B1 1~. 1~5 23N 4.4 457 4G. 5GY 5VS 6J9 7-5 71M 8P~ AACTN AAEDT AAEDW AAIKJ AAKOC AALRI AAOAW AAQFI AAXUO AAYFN ABBOA ABFNM ABFRF ABJNI ABMAC ABXDB ACDAQ ACGFO ACGFS ACNNM ACRLP ACZNC ADBBV ADEZE ADMUD ADTZH AEBSH AECPX AEFWE AEKER AENEX AFKWA AFTJW AGHFR AGUBO AGYEJ AHJVU AHZHX AIALX AIEXJ AIKHN AITUG AJOXV AKRWK ALMA_UNASSIGNED_HOLDINGS AMFUW AMRAJ AOUOD AXJTR BJAXD BKOJK BLXMC CS3 DU5 EBS EFJIC EJD EO8 EO9 EP2 EP3 F5P FDB FIRID FNPLU FYGXN G-Q GBLVA GBOLZ HZ~ IHE J1W JJJVA KOM M41 MO0 N9A O-L O9- OAUVE OZT P-8 P-9 P2P PC. Q38 RIG ROL RPZ SDF SDG SES SPC SPCBC SST SSV SSZ T5K UNMZH ~G- 9DU AATTM AAXKI AAYWO AAYXX ABWVN ACLOT ACRPL ACVFH ADCNI ADNMO AEIPS AEUPX AFJKZ AFPUW AIGII AIIUN AKBMS AKYEP ANKPU APXCP CITATION EFKBS EFLBG ~HD |
| ID | FETCH-LOGICAL-c251t-784d48c17913a3d1219762fcacd146521c7cfdba318cc2cd5ea4bd47194b115a3 |
| ISICitedReferencesCount | 1 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001346404400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1746-8094 |
| IngestDate | Sat Nov 29 02:51:27 EST 2025 Sat Aug 17 15:43:32 EDT 2024 |
| IsPeerReviewed | true |
| IsScholarly | true |
| Keywords | Functional connectivity fMRI Multimodal emotion recognition Brain signal classification Emotion valence |
| Language | English |
| LinkModel | OpenURL |
| MergedId | FETCHMERGED-LOGICAL-c251t-784d48c17913a3d1219762fcacd146521c7cfdba318cc2cd5ea4bd47194b115a3 |
| ORCID | 0000-0003-1147-6170 |
| ParticipantIDs | crossref_primary_10_1016_j_bspc_2024_106635 elsevier_sciencedirect_doi_10_1016_j_bspc_2024_106635 |
| PublicationCentury | 2000 |
| PublicationDate | October 2024 2024-10-00 |
| PublicationDateYYYYMMDD | 2024-10-01 |
| PublicationDate_xml | – month: 10 year: 2024 text: October 2024 |
| PublicationDecade | 2020 |
| PublicationTitle | Biomedical signal processing and control |
| PublicationYear | 2024 |
| Publisher | Elsevier Ltd |
| Publisher_xml | – name: Elsevier Ltd |
| References | Li, Zhang, Jiang, Zhou, Xu, Zou (b0045) 2022; 16 Döllinger, Högman, Laukka, Bänziger, Makower, Fischer, Hau (b0035) 2023; 14 Gu, Cao, Liu (b0080) 2019; 203 Peelen, Atkinson, Vuilleumier (b0010) 2010; 30 Tao, Lu (b0160) 2020 Honey, Sporns, Cammoun, Gigandet, Thiran, Meuli, Hagmann (b0050) 2009; 106 Saarimäki, Gotsopoulos, Jääskeläinen, Lampinen, Vuilleumier, Hari, Sams, Nummenmaa (b0120) 2016; 26 Kim, Schultz, Rohe, Wallraven, Lee, Bülthoff (b0005) 2015; 35 Wu, Zheng, Li, Lu (b0130) 2022; 19 Xie, Sidulova, Park (b0020) 2021; 21 Lu, Chen, Cui, Guo, Pang, Luo, Yu, Chen, Gao, Sheng, Tang, Zeng, Jiang, Gao, He, Chen (b0125) 2023; 33 Paquelet, Carrion, Lacefield, Zhou, Hen, Miller (b0155) 2022; 110 Assari, Boyce, Bazargan (b0100) 2020; 10 Gu, Liu, Li, Wang, Wang (b0140) 2020; 14 Guinamard, Clément, Goemaere, Mary, Riquet, Dellacherie (b0095) 2022; 16 Huan, Shu, Bao, Liang, Chen, Chi (b0025) 2021; 80 Cao, Liang, Yoshida, Guan (b0070) 2019; 14 Kunz, Chen, Lautenbacher, Vachon-Presseau, Rainville (b0085) 2011; 31 Yi, Leonard, Chang (b0090) 2019; 102 Fritze, Sambataro, Kubera, Brandt, Meyer-Lindenberg, Wolf, Hirjak (b0105) 2022; 272 J. Shen, J. Zheng, X. Wang, MMTrans-MT: A Framework for multimodal emotion recognition using multitask learning, in: 2021 13th International Conference on Advanced Computational Intelligence (ICACI), pp. 52–59, doi: 10.1109/ICACI52617.2021.9435906. Cheng, Xue, Dong, Hu, Zhou, Li, Huang, Lu, Yuan, Yu (b0110) 2022; 16 Li, Xu, Wang, Fang, Ji (b0075) 2020; 28 Norman, Polyn, Detre, Haxby (b0065) 2006; 10 Yu, Yin, Kaiser, Xu, Guo, Liu, Li, Fan (b0115) 2023; 100 Kim, Shinkareva, Wedell (b0015) 2017; 148 Fan, Sun, Lang, Hu, Hameed, Wei, Zhuang, Cai, Liu, Mao, Feng, Pan (b0055) 2021; 42 Yan, Chen, Li, Castellanos, Bai, Bo, Cao, Chen, Chen, Chen, Cheng, Cheng, Cui, Duan, Fang, Gong, Guo, Hou, Hu, Zang (b0060) 2019; 116 Grundei, Schmidt, Blankenburg (b0135) 2023; 44 Xu, Dong, Li, Wang, Guo, Wei, Dang (b0040) 2021; 469 Habig, Krämer, Lautenschläger, Walter, Best (b0145) 2022; 228 S. Zheng, Introduction to Cognitive Neuroscience, Peking University Press, 2010. Xiao, Wang, Hosseinzadeh Kassani, Zhang, Bai, Stephen, Wilson, Calhoun, Wang (b0165) 2019 10.1016/j.bspc.2024.106635_b0150 10.1016/j.bspc.2024.106635_b0030 Cheng (10.1016/j.bspc.2024.106635_b0110) 2022; 16 Habig (10.1016/j.bspc.2024.106635_b0145) 2022; 228 Honey (10.1016/j.bspc.2024.106635_b0050) 2009; 106 Grundei (10.1016/j.bspc.2024.106635_b0135) 2023; 44 Gu (10.1016/j.bspc.2024.106635_b0080) 2019; 203 Cao (10.1016/j.bspc.2024.106635_b0070) 2019; 14 Norman (10.1016/j.bspc.2024.106635_b0065) 2006; 10 Yan (10.1016/j.bspc.2024.106635_b0060) 2019; 116 Huan (10.1016/j.bspc.2024.106635_b0025) 2021; 80 Saarimäki (10.1016/j.bspc.2024.106635_b0120) 2016; 26 Peelen (10.1016/j.bspc.2024.106635_b0010) 2010; 30 Li (10.1016/j.bspc.2024.106635_b0045) 2022; 16 Gu (10.1016/j.bspc.2024.106635_b0140) 2020; 14 Kunz (10.1016/j.bspc.2024.106635_b0085) 2011; 31 Fritze (10.1016/j.bspc.2024.106635_b0105) 2022; 272 Döllinger (10.1016/j.bspc.2024.106635_b0035) 2023; 14 Kim (10.1016/j.bspc.2024.106635_b0015) 2017; 148 Xu (10.1016/j.bspc.2024.106635_b0040) 2021; 469 Li (10.1016/j.bspc.2024.106635_b0075) 2020; 28 Yu (10.1016/j.bspc.2024.106635_b0115) 2023; 100 Kim (10.1016/j.bspc.2024.106635_b0005) 2015; 35 Xie (10.1016/j.bspc.2024.106635_b0020) 2021; 21 Fan (10.1016/j.bspc.2024.106635_b0055) 2021; 42 Guinamard (10.1016/j.bspc.2024.106635_b0095) 2022; 16 Xiao (10.1016/j.bspc.2024.106635_b0165) 2019 Yi (10.1016/j.bspc.2024.106635_b0090) 2019; 102 Assari (10.1016/j.bspc.2024.106635_b0100) 2020; 10 Wu (10.1016/j.bspc.2024.106635_b0130) 2022; 19 Lu (10.1016/j.bspc.2024.106635_b0125) 2023; 33 Tao (10.1016/j.bspc.2024.106635_b0160) 2020 Paquelet (10.1016/j.bspc.2024.106635_b0155) 2022; 110 |
| References_xml | – volume: 203 year: 2019 ident: b0080 article-title: Modality-general representations of valences perceived from visual and auditory modalities publication-title: Neuroimage – volume: 14 start-page: 1908 year: 2020 end-page: 1920 ident: b0140 article-title: Cross-modal representations in early visual and auditory cortices revealed by multi-voxel pattern analysis publication-title: Brain Imaging Behav. – volume: 102 start-page: 1096 year: 2019 end-page: 1110 ident: b0090 article-title: The encoding of speech sounds in the superior temporal gyrus publication-title: Neuron – volume: 228 start-page: 433 year: 2022 end-page: 447 ident: b0145 article-title: Processing of sensory, painful and vestibular stimuli in the thalamus publication-title: Brain Struct. Funct. – volume: 148 start-page: 42 year: 2017 end-page: 54 ident: b0015 article-title: Representations of modality-general valence for videos and music derived from fMRI data publication-title: Neuroimage – volume: 16 start-page: 930 year: 2022 end-page: 938 ident: b0110 article-title: Abnormal functional connectivity of the salience network in insomnia publication-title: Brain Imaging Behav. – reference: S. Zheng, Introduction to Cognitive Neuroscience, Peking University Press, 2010. – volume: 14 year: 2023 ident: b0035 article-title: Trainee psychotherapists’ emotion recognition accuracy improves after training: emotion recognition training as a tool for psychotherapy education publication-title: Front. Psychol. – volume: 116 start-page: 9078 year: 2019 end-page: 9083 ident: b0060 article-title: Reduced default mode network functional connectivity in patients with recurrent major depressive disorder publication-title: Proc. Natl. Acad. Sci. – volume: 21 year: 2021 ident: b0020 article-title: Robust multimodal emotion recognition from conversation with transformer-based crossmodality fusion publication-title: Sensors – volume: 14 start-page: 475 year: 2019 end-page: 488 ident: b0070 article-title: Facial expression decoding based on fMRI brain signal publication-title: Int. J. Comput. Commun. Control. – volume: 16 year: 2022 ident: b0095 article-title: Musical abilities in children with developmental cerebellar anomalies publication-title: Front. Syst. Neurosci. – start-page: 1 year: 2020 end-page: 8 ident: b0160 article-title: Emotion recognition under sleep deprivation using a multimodal residual LSTM network publication-title: 2020 International Joint Conference on Neural Networks (IJCNN) – volume: 19 year: 2022 ident: b0130 article-title: Investigating EEG-based functional connectivity patterns for multimodal emotion recognition publication-title: J. Neural Eng. – volume: 469 start-page: 46 year: 2021 end-page: 58 ident: b0040 article-title: Weighted RSA: an improved framework on the perception of audio-visual affective speech in left insula and superior temporal gyrus publication-title: Neuroscience – volume: 28 start-page: 2615 year: 2020 end-page: 2626 ident: b0075 article-title: A multi-scale fusion convolutional neural network based on attention mechanism for the visualization analysis of EEG signals decoding publication-title: IEEE Trans. Neural Syst. Rehabil. Eng. – volume: 110 start-page: 2664 year: 2022 end-page: 2679.e8 ident: b0155 article-title: Single-cell activity and network properties of dorsal raphe nucleus serotonin neurons during emotionally salient behaviors publication-title: Neuron – volume: 272 start-page: 1097 year: 2022 end-page: 1108 ident: b0105 article-title: Characterizing the sensorimotor domain in schizophrenia spectrum disorders publication-title: Eur. Arch. Psychiatry Clin. Neurosci. – volume: 30 start-page: 10127 year: 2010 end-page: 10134 ident: b0010 article-title: Supramodal representations of perceived emotions in the human brain publication-title: J. Neurosci. – volume: 80 start-page: 8213 year: 2021 end-page: 8240 ident: b0025 article-title: Video multimodal emotion recognition based on Bi-GRU and attention fusion publication-title: Multimed. Tools Appl. – volume: 44 start-page: 5871 year: 2023 end-page: 5891 ident: b0135 article-title: A multimodal cortical network of sensory expectation violation revealed by fMRI publication-title: Hum. Brain Mapp. – volume: 10 start-page: 1364 year: 2006 end-page: 6613 ident: b0065 article-title: Beyond mind-reading: multi-voxel pattern analysis of fMRI data publication-title: Trends Cogn. Sci. – start-page: 1 year: 2019 ident: b0165 article-title: Multi-hypergraph learning based brain functional connectivity analysis in fMRI data publication-title: IEEE Trans. Med. Imaging – volume: 106 start-page: 2035 year: 2009 end-page: 2040 ident: b0050 article-title: Predicting human resting-state functional connectivity from structural connectivity publication-title: Proc. Natl. Acad. Sci. – volume: 31 start-page: 8730 year: 2011 end-page: 8738 ident: b0085 article-title: Cerebral regulation of facial expressions of pain publication-title: J. Neurosci. – volume: 26 start-page: 2563 year: 2016 end-page: 2573 ident: b0120 article-title: Discrete neural signatures of basic emotions publication-title: Cereb. Cortex – volume: 16 year: 2022 ident: b0045 article-title: Source localization and functional network analysis in emotion cognitive reappraisal with EEG-fMRI integration publication-title: Front. Hum. Neurosci. – volume: 35 start-page: 5655 year: 2015 end-page: 5663 ident: b0005 article-title: Abstract representations of associated emotions in the human brain publication-title: J. Neurosci. – volume: 10 year: 2020 ident: b0100 article-title: Nucleus accumbens functional connectivity with the frontoparietal network predicts subsequent change in body mass index for American children publication-title: Brain Sci. – volume: 42 start-page: 2353 year: 2021 end-page: 2361 ident: b0055 article-title: Diagnosis and surgical treatment of non-lesional temporal lobe epilepsy with unilateral amygdala enlargement publication-title: Neurol. Sci. – volume: 33 year: 2023 ident: b0125 article-title: Shared and distinct patterns of dynamic functional connectivity variability of thalamo-cortical circuit in bipolar depression and major depressive disorder publication-title: Cereb. Cortex – reference: J. Shen, J. Zheng, X. Wang, MMTrans-MT: A Framework for multimodal emotion recognition using multitask learning, in: 2021 13th International Conference on Advanced Computational Intelligence (ICACI), pp. 52–59, doi: 10.1109/ICACI52617.2021.9435906. – volume: 100 start-page: e616 year: 2023 end-page: e626 ident: b0115 article-title: Pathway-specific mediation effect between structure, function, and motor impairment after subcortical stroke publication-title: Neurology – volume: 14 start-page: 475 year: 2019 ident: 10.1016/j.bspc.2024.106635_b0070 article-title: Facial expression decoding based on fMRI brain signal publication-title: Int. J. Comput. Commun. Control. doi: 10.15837/ijccc.2019.4.3433 – volume: 28 start-page: 2615 issue: 12 year: 2020 ident: 10.1016/j.bspc.2024.106635_b0075 article-title: A multi-scale fusion convolutional neural network based on attention mechanism for the visualization analysis of EEG signals decoding publication-title: IEEE Trans. Neural Syst. Rehabil. Eng. doi: 10.1109/TNSRE.2020.3037326 – volume: 272 start-page: 1097 issue: 6 year: 2022 ident: 10.1016/j.bspc.2024.106635_b0105 article-title: Characterizing the sensorimotor domain in schizophrenia spectrum disorders publication-title: Eur. Arch. Psychiatry Clin. Neurosci. doi: 10.1007/s00406-021-01354-9 – volume: 16 start-page: 930 issue: 2 year: 2022 ident: 10.1016/j.bspc.2024.106635_b0110 article-title: Abnormal functional connectivity of the salience network in insomnia publication-title: Brain Imaging Behav. doi: 10.1007/s11682-021-00567-9 – volume: 44 start-page: 5871 issue: 17 year: 2023 ident: 10.1016/j.bspc.2024.106635_b0135 article-title: A multimodal cortical network of sensory expectation violation revealed by fMRI publication-title: Hum. Brain Mapp. doi: 10.1002/hbm.26482 – volume: 80 start-page: 8213 issue: 6 year: 2021 ident: 10.1016/j.bspc.2024.106635_b0025 article-title: Video multimodal emotion recognition based on Bi-GRU and attention fusion publication-title: Multimed. Tools Appl. doi: 10.1007/s11042-020-10030-4 – volume: 35 start-page: 5655 issue: 14 year: 2015 ident: 10.1016/j.bspc.2024.106635_b0005 article-title: Abstract representations of associated emotions in the human brain publication-title: J. Neurosci. doi: 10.1523/JNEUROSCI.4059-14.2015 – ident: 10.1016/j.bspc.2024.106635_b0030 doi: 10.1109/ICACI52617.2021.9435906 – volume: 30 start-page: 10127 issue: 30 year: 2010 ident: 10.1016/j.bspc.2024.106635_b0010 article-title: Supramodal representations of perceived emotions in the human brain publication-title: J. Neurosci. doi: 10.1523/JNEUROSCI.2161-10.2010 – volume: 100 start-page: e616 issue: 6 year: 2023 ident: 10.1016/j.bspc.2024.106635_b0115 article-title: Pathway-specific mediation effect between structure, function, and motor impairment after subcortical stroke publication-title: Neurology doi: 10.1212/WNL.0000000000201495 – volume: 102 start-page: 1096 issue: 6 year: 2019 ident: 10.1016/j.bspc.2024.106635_b0090 article-title: The encoding of speech sounds in the superior temporal gyrus publication-title: Neuron doi: 10.1016/j.neuron.2019.04.023 – volume: 21 issue: 14 year: 2021 ident: 10.1016/j.bspc.2024.106635_b0020 article-title: Robust multimodal emotion recognition from conversation with transformer-based crossmodality fusion publication-title: Sensors doi: 10.3390/s21144913 – volume: 31 start-page: 8730 issue: 24 year: 2011 ident: 10.1016/j.bspc.2024.106635_b0085 article-title: Cerebral regulation of facial expressions of pain publication-title: J. Neurosci. doi: 10.1523/JNEUROSCI.0217-11.2011 – volume: 33 year: 2023 ident: 10.1016/j.bspc.2024.106635_b0125 article-title: Shared and distinct patterns of dynamic functional connectivity variability of thalamo-cortical circuit in bipolar depression and major depressive disorder publication-title: Cereb. Cortex doi: 10.1093/cercor/bhac534 – volume: 42 start-page: 2353 issue: 6 year: 2021 ident: 10.1016/j.bspc.2024.106635_b0055 article-title: Diagnosis and surgical treatment of non-lesional temporal lobe epilepsy with unilateral amygdala enlargement publication-title: Neurol. Sci. doi: 10.1007/s10072-020-04794-8 – volume: 10 issue: 10 year: 2020 ident: 10.1016/j.bspc.2024.106635_b0100 article-title: Nucleus accumbens functional connectivity with the frontoparietal network predicts subsequent change in body mass index for American children publication-title: Brain Sci. doi: 10.3390/brainsci10100703 – volume: 203 year: 2019 ident: 10.1016/j.bspc.2024.106635_b0080 article-title: Modality-general representations of valences perceived from visual and auditory modalities publication-title: Neuroimage doi: 10.1016/j.neuroimage.2019.116199 – volume: 110 start-page: 2664 issue: 16 year: 2022 ident: 10.1016/j.bspc.2024.106635_b0155 article-title: Single-cell activity and network properties of dorsal raphe nucleus serotonin neurons during emotionally salient behaviors publication-title: Neuron doi: 10.1016/j.neuron.2022.05.015 – volume: 14 year: 2023 ident: 10.1016/j.bspc.2024.106635_b0035 article-title: Trainee psychotherapists’ emotion recognition accuracy improves after training: emotion recognition training as a tool for psychotherapy education publication-title: Front. Psychol. doi: 10.3389/fpsyg.2023.1188634 – volume: 116 start-page: 9078 issue: 18 year: 2019 ident: 10.1016/j.bspc.2024.106635_b0060 article-title: Reduced default mode network functional connectivity in patients with recurrent major depressive disorder publication-title: Proc. Natl. Acad. Sci. doi: 10.1073/pnas.1900390116 – volume: 16 year: 2022 ident: 10.1016/j.bspc.2024.106635_b0095 article-title: Musical abilities in children with developmental cerebellar anomalies publication-title: Front. Syst. Neurosci. – start-page: 1 year: 2020 ident: 10.1016/j.bspc.2024.106635_b0160 article-title: Emotion recognition under sleep deprivation using a multimodal residual LSTM network – volume: 26 start-page: 2563 issue: 6 year: 2016 ident: 10.1016/j.bspc.2024.106635_b0120 article-title: Discrete neural signatures of basic emotions publication-title: Cereb. Cortex doi: 10.1093/cercor/bhv086 – volume: 469 start-page: 46 year: 2021 ident: 10.1016/j.bspc.2024.106635_b0040 article-title: Weighted RSA: an improved framework on the perception of audio-visual affective speech in left insula and superior temporal gyrus publication-title: Neuroscience doi: 10.1016/j.neuroscience.2021.06.002 – volume: 106 start-page: 2035 issue: 6 year: 2009 ident: 10.1016/j.bspc.2024.106635_b0050 article-title: Predicting human resting-state functional connectivity from structural connectivity publication-title: Proc. Natl. Acad. Sci. doi: 10.1073/pnas.0811168106 – volume: 14 start-page: 1908 issue: 5 year: 2020 ident: 10.1016/j.bspc.2024.106635_b0140 article-title: Cross-modal representations in early visual and auditory cortices revealed by multi-voxel pattern analysis publication-title: Brain Imaging Behav. doi: 10.1007/s11682-019-00135-2 – volume: 228 start-page: 433 issue: 2 year: 2022 ident: 10.1016/j.bspc.2024.106635_b0145 article-title: Processing of sensory, painful and vestibular stimuli in the thalamus publication-title: Brain Struct. Funct. doi: 10.1007/s00429-022-02582-y – volume: 19 year: 2022 ident: 10.1016/j.bspc.2024.106635_b0130 article-title: Investigating EEG-based functional connectivity patterns for multimodal emotion recognition publication-title: J. Neural Eng. doi: 10.1088/1741-2552/ac49a7 – ident: 10.1016/j.bspc.2024.106635_b0150 – volume: 10 start-page: 1364 issue: 424–430 year: 2006 ident: 10.1016/j.bspc.2024.106635_b0065 article-title: Beyond mind-reading: multi-voxel pattern analysis of fMRI data publication-title: Trends Cogn. Sci. – volume: 148 start-page: 42 year: 2017 ident: 10.1016/j.bspc.2024.106635_b0015 article-title: Representations of modality-general valence for videos and music derived from fMRI data publication-title: Neuroimage doi: 10.1016/j.neuroimage.2017.01.002 – start-page: 1 year: 2019 ident: 10.1016/j.bspc.2024.106635_b0165 article-title: Multi-hypergraph learning based brain functional connectivity analysis in fMRI data publication-title: IEEE Trans. Med. Imaging – volume: 16 year: 2022 ident: 10.1016/j.bspc.2024.106635_b0045 article-title: Source localization and functional network analysis in emotion cognitive reappraisal with EEG-fMRI integration publication-title: Front. Hum. Neurosci. doi: 10.3389/fnhum.2022.960784 |
| SSID | ssj0048714 |
| Score | 2.3544858 |
| Snippet | •The differences in multimodal affective cognitive brain networks were further discovered.•Computer algorithms provide a more sufficient theoretical basis for... |
| SourceID | crossref elsevier |
| SourceType | Index Database Publisher |
| StartPage | 106635 |
| SubjectTerms | Brain signal classification Emotion valence fMRI Functional connectivity Multimodal emotion recognition |
| Title | Investigating the brain network characteristics of multimodal emotion recognition and its classification applications based on functional connectivity patterns |
| URI | https://dx.doi.org/10.1016/j.bspc.2024.106635 |
| Volume | 96 |
| WOSCitedRecordID | wos001346404400001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | 1 |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| journalDatabaseRights | – providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 issn: 1746-8094 databaseCode: AIEXJ dateStart: 20060101 customDbUrl: isFulltext: true dateEnd: 99991231 titleUrlDefault: https://www.sciencedirect.com omitProxy: false ssIdentifier: ssj0048714 providerName: Elsevier |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1Lb9NAEF6FlAMcEE9RXtoDtyiVbK-79rFCRYBQxaFIvln7MrgC27RJlX_DP-lv68w-7E1UED1wsaKVvbEyX3ZmZ775lpC3Wa5TruD_LTQXS6ZMg-sgCkHKspFJLhpjG4U_85OToqrKL7PZVeiFufzBu67YbMrhv5oaxsDY2Dp7C3OPk8IAfAajwxXMDtd_MnyknOE7oSQeA7HoHOEbW323JJqxwI6swp-9BnMZd6zPYiQWebYy1hcURtpILXKgiWvfC_SGGisP6Ch9flEhiUb54ykGK-Tpc4Ohimx7_11jZvsNHxlc30LonPRE-pEktLagayca0dpmeqtW9L_a8S7PMq7a7rvop6qXYxeYbrOOUx0pG0lzYXXmDNWT3anIYfku4_U3sRHUja7BZSnODuTFgNKVKTuYbt7W4d7xjyNrMRDizmqco8Y5ajfHHbKX8rws5mTv6ONx9SnEArAbtOry44v7ti3HMNx9k5tDoyjcOX1IHvh9Cj1y-HpEZqZ7TO5H6pVPyO8tpFFAGrVIox5pdAdptG_ohDTqkUYjpFGwOQWk0W2k0Rhp1CKNwuiENBojjQakPSVf3x-fvvuw9Od9LBVE2aslL5hmhULB3ExkOgFnCq66UUJp8OcQZyquGi0FuCGlUqVzI5jUEF2VTMLGRmTPyLzrO_Oc0MM80ULl4LxSzRQ7LGRWmIanhVRNIkSyTxbhp64HJ-tS_9m8-yQP1qh9YOoCzhrA9ZfnXtzqW16SexPoX5H56nxtXpO76nLVXpy_8ci6BiQ5vDc |
| linkProvider | Elsevier |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Investigating+the+brain+network+characteristics+of+multimodal+emotion+recognition+and+its+classification+applications+based+on+functional+connectivity+patterns&rft.jtitle=Biomedical+signal+processing+and+control&rft.au=Gu%2C+Jin&rft.au=Luo%2C+Xiaoqi&rft.au=Gong%2C+Xinhao&rft.au=Su%2C+Chenxu&rft.date=2024-10-01&rft.issn=1746-8094&rft.volume=96&rft.spage=106635&rft_id=info:doi/10.1016%2Fj.bspc.2024.106635&rft.externalDBID=n%2Fa&rft.externalDocID=10_1016_j_bspc_2024_106635 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1746-8094&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1746-8094&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1746-8094&client=summon |