Cracking the social code of speech prosody using reverse correlation
Human listeners excel at forming high-level social representations about each other, even from the briefest of utterances. In particular, pitch is widely recognized as the auditory dimension that conveys most of the information about a speaker's traits, emotional states, and attitudes. While pa...
Saved in:
| Published in: | Proceedings of the National Academy of Sciences - PNAS Vol. 115; no. 15; p. 3972 |
|---|---|
| Main Authors: | , , , |
| Format: | Journal Article |
| Language: | English |
| Published: |
United States
10.04.2018
|
| Subjects: | |
| ISSN: | 1091-6490, 1091-6490 |
| Online Access: | Get more information |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Abstract | Human listeners excel at forming high-level social representations about each other, even from the briefest of utterances. In particular, pitch is widely recognized as the auditory dimension that conveys most of the information about a speaker's traits, emotional states, and attitudes. While past research has primarily looked at the influence of mean pitch, almost nothing is known about how intonation patterns, i.e., finely tuned pitch trajectories around the mean, may determine social judgments in speech. Here, we introduce an experimental paradigm that combines state-of-the-art voice transformation algorithms with psychophysical reverse correlation and show that two of the most important dimensions of social judgments, a speaker's perceived dominance and trustworthiness, are driven by robust and distinguishing pitch trajectories in short utterances like the word "Hello," which remained remarkably stable whether male or female listeners judged male or female speakers. These findings reveal a unique communicative adaptation that enables listeners to infer social traits regardless of speakers' physical characteristics, such as sex and mean pitch. By characterizing how any given individual's mental representations may differ from this generic code, the method introduced here opens avenues to explore dysprosody and social-cognitive deficits in disorders like autism spectrum and schizophrenia. In addition, once derived experimentally, these prototypes can be applied to novel utterances, thus providing a principled way to modulate personality impressions in arbitrary speech signals. |
|---|---|
| AbstractList | Human listeners excel at forming high-level social representations about each other, even from the briefest of utterances. In particular, pitch is widely recognized as the auditory dimension that conveys most of the information about a speaker's traits, emotional states, and attitudes. While past research has primarily looked at the influence of mean pitch, almost nothing is known about how intonation patterns, i.e., finely tuned pitch trajectories around the mean, may determine social judgments in speech. Here, we introduce an experimental paradigm that combines state-of-the-art voice transformation algorithms with psychophysical reverse correlation and show that two of the most important dimensions of social judgments, a speaker's perceived dominance and trustworthiness, are driven by robust and distinguishing pitch trajectories in short utterances like the word "Hello," which remained remarkably stable whether male or female listeners judged male or female speakers. These findings reveal a unique communicative adaptation that enables listeners to infer social traits regardless of speakers' physical characteristics, such as sex and mean pitch. By characterizing how any given individual's mental representations may differ from this generic code, the method introduced here opens avenues to explore dysprosody and social-cognitive deficits in disorders like autism spectrum and schizophrenia. In addition, once derived experimentally, these prototypes can be applied to novel utterances, thus providing a principled way to modulate personality impressions in arbitrary speech signals.Human listeners excel at forming high-level social representations about each other, even from the briefest of utterances. In particular, pitch is widely recognized as the auditory dimension that conveys most of the information about a speaker's traits, emotional states, and attitudes. While past research has primarily looked at the influence of mean pitch, almost nothing is known about how intonation patterns, i.e., finely tuned pitch trajectories around the mean, may determine social judgments in speech. Here, we introduce an experimental paradigm that combines state-of-the-art voice transformation algorithms with psychophysical reverse correlation and show that two of the most important dimensions of social judgments, a speaker's perceived dominance and trustworthiness, are driven by robust and distinguishing pitch trajectories in short utterances like the word "Hello," which remained remarkably stable whether male or female listeners judged male or female speakers. These findings reveal a unique communicative adaptation that enables listeners to infer social traits regardless of speakers' physical characteristics, such as sex and mean pitch. By characterizing how any given individual's mental representations may differ from this generic code, the method introduced here opens avenues to explore dysprosody and social-cognitive deficits in disorders like autism spectrum and schizophrenia. In addition, once derived experimentally, these prototypes can be applied to novel utterances, thus providing a principled way to modulate personality impressions in arbitrary speech signals. Human listeners excel at forming high-level social representations about each other, even from the briefest of utterances. In particular, pitch is widely recognized as the auditory dimension that conveys most of the information about a speaker's traits, emotional states, and attitudes. While past research has primarily looked at the influence of mean pitch, almost nothing is known about how intonation patterns, i.e., finely tuned pitch trajectories around the mean, may determine social judgments in speech. Here, we introduce an experimental paradigm that combines state-of-the-art voice transformation algorithms with psychophysical reverse correlation and show that two of the most important dimensions of social judgments, a speaker's perceived dominance and trustworthiness, are driven by robust and distinguishing pitch trajectories in short utterances like the word "Hello," which remained remarkably stable whether male or female listeners judged male or female speakers. These findings reveal a unique communicative adaptation that enables listeners to infer social traits regardless of speakers' physical characteristics, such as sex and mean pitch. By characterizing how any given individual's mental representations may differ from this generic code, the method introduced here opens avenues to explore dysprosody and social-cognitive deficits in disorders like autism spectrum and schizophrenia. In addition, once derived experimentally, these prototypes can be applied to novel utterances, thus providing a principled way to modulate personality impressions in arbitrary speech signals. |
| Author | Aucouturier, Jean-Julien Belin, Pascal Burred, Juan José Ponsot, Emmanuel |
| Author_xml | – sequence: 1 givenname: Emmanuel surname: Ponsot fullname: Ponsot, Emmanuel email: emmanuel.ponsot@ens.fr organization: Département d'Études Cognitives, Ecole Normale Supérieure, Paris Sciences et Lettres Research University, 75005 Paris, France – sequence: 2 givenname: Juan José surname: Burred fullname: Burred, Juan José organization: Independent Researcher, 75013 Paris, France – sequence: 3 givenname: Pascal orcidid: 0000-0002-7578-6365 surname: Belin fullname: Belin, Pascal organization: Département de Psychologie, Université de Montréal, Montréal, QC H3T 1J4, Canada – sequence: 4 givenname: Jean-Julien orcidid: 0000-0002-4477-4812 surname: Aucouturier fullname: Aucouturier, Jean-Julien organization: Okanoya Emotional Information Project, Exploratory Research for Advanced Technology, Japan Science and Technology Agency, Wako, Saitama 332-0012, Japan |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/29581266$$D View this record in MEDLINE/PubMed |
| BookMark | eNpNj7tPwzAYxC1URB8wsyGPLCmfXT_iEZXykCqxwBw59hcaSONgJ0j97wmiSOiGu-Gn092cTNrQIiGXDJYM9Oqma21aMs0UGGBMnpAZA8MyJQxM_uUpmaf0DgBG5nBGpnx0xpWakbt1tO6jbt9ov0OagqttQ13wSENFU4fodrSLIQV_oEP64SJ-YUw4QjFiY_s6tOfktLJNwoujL8jr_eZl_Zhtnx-e1rfbzElt-qzMKw_SCFcpjlJrBSvLUMMobldY5VJygV4Y5pSTSoEvma6YVB5MCbnnC3L92zsu-hww9cW-Tg6bxrYYhlRwYAaEULkY0asjOpR79EUX672Nh-LvOf8Gov5cLw |
| CitedBy_id | crossref_primary_10_1111_brv_13131 crossref_primary_10_1177_17470218211010144 crossref_primary_10_1073_pnas_2400369121 crossref_primary_10_1038_s41467_020_20649_4 crossref_primary_10_1016_j_cub_2023_12_001 crossref_primary_10_3389_fpsyg_2022_1028106 crossref_primary_10_1038_s41598_024_64295_y crossref_primary_10_1073_pnas_1806857115 crossref_primary_10_1371_journal_pone_0210555 crossref_primary_10_1371_journal_pone_0204991 crossref_primary_10_1371_journal_pone_0285028 crossref_primary_10_1162_imag_a_00003 crossref_primary_10_1177_1754073920934544 crossref_primary_10_1007_s42761_023_00215_z crossref_primary_10_1016_j_jbusres_2025_115282 crossref_primary_10_1038_s41598_022_18838_w crossref_primary_10_1098_rsos_211496 crossref_primary_10_3758_s13428_023_02281_4 crossref_primary_10_1007_s40750_020_00154_0 crossref_primary_10_1016_j_jpain_2023_08_007 crossref_primary_10_1111_1460_6984_12909 crossref_primary_10_1111_bjop_12437 crossref_primary_10_3758_s13428_023_02098_1 crossref_primary_10_1080_13803395_2024_2348212 crossref_primary_10_1093_beheco_arz173 crossref_primary_10_1177_17456916231217722 crossref_primary_10_1177_2041669519861981 crossref_primary_10_1016_j_clinph_2021_12_015 crossref_primary_10_1038_s41562_020_00987_5 crossref_primary_10_1016_j_neucli_2025_103050 crossref_primary_10_1177_10298649241255694 crossref_primary_10_1016_j_cortex_2024_06_016 crossref_primary_10_1093_cercor_bhaa175 crossref_primary_10_1016_j_cub_2018_11_049 crossref_primary_10_1177_1754073919898522 crossref_primary_10_3389_fpsyg_2025_1495456 crossref_primary_10_3758_s13428_022_01946_w crossref_primary_10_1016_j_reia_2025_202599 crossref_primary_10_1002_wcs_1579 crossref_primary_10_1016_j_paid_2020_110389 crossref_primary_10_1177_14747049221108883 crossref_primary_10_1073_pnas_1806345115 crossref_primary_10_1121_10_0037102 crossref_primary_10_1371_journal_pone_0205943 crossref_primary_10_1093_brain_awac454 crossref_primary_10_1080_13803395_2024_2428728 crossref_primary_10_1177_13623613221111207 crossref_primary_10_1038_s41597_025_05267_3 crossref_primary_10_1093_jole_lzaa009 crossref_primary_10_1371_journal_pone_0290612 crossref_primary_10_1371_journal_pone_0232431 crossref_primary_10_1016_j_cobeha_2021_101100 crossref_primary_10_1109_TMM_2021_3071243 |
| ContentType | Journal Article |
| Copyright | Copyright © 2018 the Author(s). Published by PNAS. |
| Copyright_xml | – notice: Copyright © 2018 the Author(s). Published by PNAS. |
| DBID | CGR CUY CVF ECM EIF NPM 7X8 |
| DOI | 10.1073/pnas.1716090115 |
| DatabaseName | Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed MEDLINE - Academic |
| DatabaseTitle | MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) MEDLINE - Academic |
| DatabaseTitleList | MEDLINE - Academic MEDLINE |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | no_fulltext_linktorsrc |
| Discipline | Sciences (General) |
| EISSN | 1091-6490 |
| ExternalDocumentID | 29581266 |
| Genre | Research Support, Non-U.S. Gov't Journal Article |
| GrantInformation_xml | – fundername: European Research Council grantid: 335536 |
| GroupedDBID | --- -DZ -~X .55 0R~ 123 29P 2AX 2FS 2WC 4.4 53G 5RE 5VS 85S AACGO AAFWJ AANCE ABBHK ABOCM ABPLY ABPPZ ABTLG ABXSQ ABZEH ACGOD ACHIC ACIWK ACNCT ACPRK ADQXQ ADULT AENEX AEUPB AEXZC AFFNX AFOSN AFRAH ALMA_UNASSIGNED_HOLDINGS AQVQM BKOMP CGR CS3 CUY CVF D0L DCCCD DIK DU5 E3Z EBS ECM EIF EJD F5P FRP GX1 H13 HH5 HYE IPSME JAAYA JBMMH JENOY JHFFW JKQEH JLS JLXEF JPM JSG JST KQ8 L7B LU7 N9A NPM N~3 O9- OK1 PNE PQQKQ R.V RHI RNA RNS RPM RXW SA0 SJN TAE TN5 UKR W8F WH7 WOQ WOW X7M XSW Y6R YBH YKV YSK ZCA ~02 ~KM 7X8 |
| ID | FETCH-LOGICAL-c579t-b8fd0594cf62e577603a1e707072a3ef85524ed491c6c5660db17f156d09b08d2 |
| IEDL.DBID | 7X8 |
| ISICitedReferencesCount | 72 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000429540300077&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1091-6490 |
| IngestDate | Thu Oct 02 06:47:48 EDT 2025 Sat May 31 02:09:26 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 15 |
| Keywords | voice reverse-correlation prosody social traits speech |
| Language | English |
| License | Copyright © 2018 the Author(s). Published by PNAS. |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-c579t-b8fd0594cf62e577603a1e707072a3ef85524ed491c6c5660db17f156d09b08d2 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| ORCID | 0000-0002-7578-6365 0000-0002-4477-4812 |
| OpenAccessLink | https://pubmed.ncbi.nlm.nih.gov/PMC5899438 |
| PMID | 29581266 |
| PQID | 2019044684 |
| PQPubID | 23479 |
| ParticipantIDs | proquest_miscellaneous_2019044684 pubmed_primary_29581266 |
| PublicationCentury | 2000 |
| PublicationDate | 2018-04-10 |
| PublicationDateYYYYMMDD | 2018-04-10 |
| PublicationDate_xml | – month: 04 year: 2018 text: 2018-04-10 day: 10 |
| PublicationDecade | 2010 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | Proceedings of the National Academy of Sciences - PNAS |
| PublicationTitleAlternate | Proc Natl Acad Sci U S A |
| PublicationYear | 2018 |
| References | 29899154 - Proc Natl Acad Sci U S A. 2018 Jul 3;115(27):E6103. doi: 10.1073/pnas.1806345115. 29899153 - Proc Natl Acad Sci U S A. 2018 Jul 3;115(27):E6104-E6105. doi: 10.1073/pnas.1806857115. |
| References_xml | – reference: 29899153 - Proc Natl Acad Sci U S A. 2018 Jul 3;115(27):E6104-E6105. doi: 10.1073/pnas.1806857115. – reference: 29899154 - Proc Natl Acad Sci U S A. 2018 Jul 3;115(27):E6103. doi: 10.1073/pnas.1806345115. |
| SSID | ssj0009580 |
| Score | 2.5301068 |
| Snippet | Human listeners excel at forming high-level social representations about each other, even from the briefest of utterances. In particular, pitch is widely... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| StartPage | 3972 |
| SubjectTerms | Adult Auditory Perception Emotions Female Humans Judgment Male Middle Aged Random Allocation Social Behavior Speech Speech Perception Trust Voice Young Adult |
| Title | Cracking the social code of speech prosody using reverse correlation |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/29581266 https://www.proquest.com/docview/2019044684 |
| Volume | 115 |
| WOSCitedRecordID | wos000429540300077&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8QwEA7qevCirs_1RQQPeoimTdqmJ5HVxYvLHhT2tqRJql7a2q6C_96ZNoteBMFLoaWFMJ3JfPPIN4ScSQvPhTAssCpn0ljBkIWMCY20uDIWmebtsIlkPFbTaTrxCbfGt1Uu9sR2o7alwRw5BOnguiB2UfK6emM4NQqrq36ExjLpCYAyaJjJVP0g3VUdG0EasFimfEHtk4irqtDNJVLFcDx7Gf2OL1s_M9r47wo3ybpHmPSmU4k-WXLFFul7G27ouSeavtgmt8NaG0yVU0CBtMueUzzjTsucNpVz5oXCGprSflLsj3-myPdUNw5eqmvfRbdDnkZ3j8N75qcqMBMl6ZxlKrdI0mLyOHRRksRc6MAlSPsTauFyFUWhdFamgYkNgD1usyDJIcyzPM24suEuWSnKwu0TCo7OKKmtBuwrIdRSEfrDzFoc9acCMSCnC0nNQGuxFKELV743s29ZDcheJ-5Z1dFrzEL4bQHghoM_fH1I1uBGYXkn4Eekl4PNumOyaj7mr0190qoDXMeThy8As72r |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Cracking+the+social+code+of+speech+prosody+using+reverse+correlation&rft.jtitle=Proceedings+of+the+National+Academy+of+Sciences+-+PNAS&rft.au=Ponsot%2C+Emmanuel&rft.au=Burred%2C+Juan+Jos%C3%A9&rft.au=Belin%2C+Pascal&rft.au=Aucouturier%2C+Jean-Julien&rft.date=2018-04-10&rft.eissn=1091-6490&rft.volume=115&rft.issue=15&rft.spage=3972&rft_id=info:doi/10.1073%2Fpnas.1716090115&rft_id=info%3Apmid%2F29581266&rft_id=info%3Apmid%2F29581266&rft.externalDocID=29581266 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1091-6490&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1091-6490&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1091-6490&client=summon |