The psychology of intelligence analysis: drivers of prediction accuracy in world politics
This article extends psychological methods and concepts into a domain that is as profoundly consequential as it is poorly understood: intelligence analysis. We report findings from a geopolitical forecasting tournament that assessed the accuracy of more than 150,000 forecasts of 743 participants on...
Gespeichert in:
| Veröffentlicht in: | Journal of experimental psychology. Applied Jg. 21; H. 1; S. 1 |
|---|---|
| Hauptverfasser: | , , , , , , , , , |
| Format: | Journal Article |
| Sprache: | Englisch |
| Veröffentlicht: |
United States
01.03.2015
|
| Schlagworte: | |
| ISSN: | 1939-2192, 1939-2192 |
| Online-Zugang: | Weitere Angaben |
| Tags: |
Tag hinzufügen
Keine Tags, Fügen Sie den ersten Tag hinzu!
|
| Abstract | This article extends psychological methods and concepts into a domain that is as profoundly consequential as it is poorly understood: intelligence analysis. We report findings from a geopolitical forecasting tournament that assessed the accuracy of more than 150,000 forecasts of 743 participants on 199 events occurring over 2 years. Participants were above average in intelligence and political knowledge relative to the general population. Individual differences in performance emerged, and forecasting skills were surprisingly consistent over time. Key predictors were (a) dispositional variables of cognitive ability, political knowledge, and open-mindedness; (b) situational variables of training in probabilistic reasoning and participation in collaborative teams that shared information and discussed rationales (Mellers, Ungar, et al., 2014); and (c) behavioral variables of deliberation time and frequency of belief updating. We developed a profile of the best forecasters; they were better at inductive reasoning, pattern detection, cognitive flexibility, and open-mindedness. They had greater understanding of geopolitics, training in probabilistic reasoning, and opportunities to succeed in cognitively enriched team environments. Last but not least, they viewed forecasting as a skill that required deliberate practice, sustained effort, and constant monitoring of current affairs. |
|---|---|
| AbstractList | This article extends psychological methods and concepts into a domain that is as profoundly consequential as it is poorly understood: intelligence analysis. We report findings from a geopolitical forecasting tournament that assessed the accuracy of more than 150,000 forecasts of 743 participants on 199 events occurring over 2 years. Participants were above average in intelligence and political knowledge relative to the general population. Individual differences in performance emerged, and forecasting skills were surprisingly consistent over time. Key predictors were (a) dispositional variables of cognitive ability, political knowledge, and open-mindedness; (b) situational variables of training in probabilistic reasoning and participation in collaborative teams that shared information and discussed rationales (Mellers, Ungar, et al., 2014); and (c) behavioral variables of deliberation time and frequency of belief updating. We developed a profile of the best forecasters; they were better at inductive reasoning, pattern detection, cognitive flexibility, and open-mindedness. They had greater understanding of geopolitics, training in probabilistic reasoning, and opportunities to succeed in cognitively enriched team environments. Last but not least, they viewed forecasting as a skill that required deliberate practice, sustained effort, and constant monitoring of current affairs. This article extends psychological methods and concepts into a domain that is as profoundly consequential as it is poorly understood: intelligence analysis. We report findings from a geopolitical forecasting tournament that assessed the accuracy of more than 150,000 forecasts of 743 participants on 199 events occurring over 2 years. Participants were above average in intelligence and political knowledge relative to the general population. Individual differences in performance emerged, and forecasting skills were surprisingly consistent over time. Key predictors were (a) dispositional variables of cognitive ability, political knowledge, and open-mindedness; (b) situational variables of training in probabilistic reasoning and participation in collaborative teams that shared information and discussed rationales (Mellers, Ungar, et al., 2014); and (c) behavioral variables of deliberation time and frequency of belief updating. We developed a profile of the best forecasters; they were better at inductive reasoning, pattern detection, cognitive flexibility, and open-mindedness. They had greater understanding of geopolitics, training in probabilistic reasoning, and opportunities to succeed in cognitively enriched team environments. Last but not least, they viewed forecasting as a skill that required deliberate practice, sustained effort, and constant monitoring of current affairs.This article extends psychological methods and concepts into a domain that is as profoundly consequential as it is poorly understood: intelligence analysis. We report findings from a geopolitical forecasting tournament that assessed the accuracy of more than 150,000 forecasts of 743 participants on 199 events occurring over 2 years. Participants were above average in intelligence and political knowledge relative to the general population. Individual differences in performance emerged, and forecasting skills were surprisingly consistent over time. Key predictors were (a) dispositional variables of cognitive ability, political knowledge, and open-mindedness; (b) situational variables of training in probabilistic reasoning and participation in collaborative teams that shared information and discussed rationales (Mellers, Ungar, et al., 2014); and (c) behavioral variables of deliberation time and frequency of belief updating. We developed a profile of the best forecasters; they were better at inductive reasoning, pattern detection, cognitive flexibility, and open-mindedness. They had greater understanding of geopolitics, training in probabilistic reasoning, and opportunities to succeed in cognitively enriched team environments. Last but not least, they viewed forecasting as a skill that required deliberate practice, sustained effort, and constant monitoring of current affairs. |
| Author | Tetlock, Philip Atanasov, Pavel Ungar, Lyle Horowitz, Michael Bishop, Michael M Metz, S Emlen Stone, Eric Rohrbaugh, Nick Merkle, Ed Mellers, Barbara |
| Author_xml | – sequence: 1 givenname: Barbara surname: Mellers fullname: Mellers, Barbara organization: Department of Psychology, University of Pennsylvania – sequence: 2 givenname: Eric surname: Stone fullname: Stone, Eric organization: Department of Psychology, University of Pennsylvania – sequence: 3 givenname: Pavel surname: Atanasov fullname: Atanasov, Pavel organization: Department of Psychology, University of Pennsylvania – sequence: 4 givenname: Nick surname: Rohrbaugh fullname: Rohrbaugh, Nick organization: Department of Psychology, University of Pennsylvania – sequence: 5 givenname: S Emlen surname: Metz fullname: Metz, S Emlen organization: Department of Psychology, University of Pennsylvania – sequence: 6 givenname: Lyle surname: Ungar fullname: Ungar, Lyle organization: Department of Computer Science, University of Pennsylvania – sequence: 7 givenname: Michael M surname: Bishop fullname: Bishop, Michael M organization: Department of Psychology, University of Pennsylvania – sequence: 8 givenname: Michael surname: Horowitz fullname: Horowitz, Michael organization: Department of Political Science, University of Pennsylvania – sequence: 9 givenname: Ed surname: Merkle fullname: Merkle, Ed organization: Department of Psychology, University of Missouri – sequence: 10 givenname: Philip surname: Tetlock fullname: Tetlock, Philip organization: Department of Psychology, University of Pennsylvania |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/25581088$$D View this record in MEDLINE/PubMed |
| BookMark | eNpNkDtPwzAUhS1URB-w8AOQR5aAH0lss6GqPKRKLGVgihzHbo1cO9gJkH9PKorEXe4dvnN1zpmDiQ9eA3CJ0Q1GlN1-yxYdJkcnYIYFFRnBgkz-3VMwT-l9RLgQ-RmYkqLgGHE-A2-bnYZtGtQuuLAdYDDQ-k47Z7faKw2ll25INt3BJtpPHdOBaKNurOps8FAq1UephlEFv0J0DWyDs51V6RycGumSvjjuBXh9WG2WT9n65fF5eb_OZI5FlzGEEZHYsJojbYoalaIUWLKGSy1IyQ0ZrVKZy6JgVAmhjBK5IajGTDMqDFmA69-_bQwfvU5dtbdJjQmk16FPFS5LklOKORrRqyPa13vdVG20exmH6q8O8gOk-GNA |
| CitedBy_id | crossref_primary_10_1002_ffo2_173 crossref_primary_10_1080_02684527_2025_2466267 crossref_primary_10_1007_s11196_025_10258_z crossref_primary_10_1177_0963721414534257 crossref_primary_10_1111_risa_13833 crossref_primary_10_1111_risa_14009 crossref_primary_10_1080_20445911_2024_2417104 crossref_primary_10_1287_mnsc_2015_2374 crossref_primary_10_1080_02684527_2019_1553375 crossref_primary_10_1002_sys_21426 crossref_primary_10_3390_jintelligence6020023 crossref_primary_10_1016_j_ijforecast_2025_01_003 crossref_primary_10_1002_arcp_1089 crossref_primary_10_3390_ejihpe10010016 crossref_primary_10_1017_XPS_2019_27 crossref_primary_10_1111_bjep_12432 crossref_primary_10_1177_1745691615598511 crossref_primary_10_1080_13662716_2023_2281983 crossref_primary_10_1007_s10664_023_10332_9 crossref_primary_10_1108_IJCMA_02_2015_0008 crossref_primary_10_1080_02684527_2016_1147164 crossref_primary_10_1111_risa_14177 crossref_primary_10_1080_13546783_2025_2520186 crossref_primary_10_1016_j_tics_2020_01_004 crossref_primary_10_3233_ISU_180009 crossref_primary_10_1177_21677026221114315 crossref_primary_10_1186_s41235_024_00560_y crossref_primary_10_3389_fpsyg_2021_662279 crossref_primary_10_1007_s12144_022_03891_5 crossref_primary_10_1111_cogs_70090 crossref_primary_10_1098_rsos_221216 crossref_primary_10_3389_fpsyg_2022_943069 crossref_primary_10_1177_17456916211059817 crossref_primary_10_1177_17456916231185339 crossref_primary_10_1111_2041_210X_14017 crossref_primary_10_1177_00018392251313737 crossref_primary_10_1002_bdm_2285 crossref_primary_10_1080_13546783_2020_1837241 crossref_primary_10_1080_10864415_2024_2401948 crossref_primary_10_2139_ssrn_3779404 crossref_primary_10_1016_j_vaccine_2022_02_054 crossref_primary_10_1002_ffo2_199 crossref_primary_10_1287_deca_2017_0362 crossref_primary_10_1038_s41598_020_72690_4 crossref_primary_10_2139_ssrn_2898954 crossref_primary_10_1186_s40798_023_00629_w crossref_primary_10_1089_big_2016_0044 crossref_primary_10_1111_poms_12664 crossref_primary_10_1080_11663081_2023_2244716 crossref_primary_10_1002_widm_1354 crossref_primary_10_1027_1864_9335_a000368 crossref_primary_10_1177_1745691615577794 crossref_primary_10_1519_JSC_0000000000004989 crossref_primary_10_1145_3707649 crossref_primary_10_1287_mnsc_2016_2525 crossref_primary_10_3389_fpsyg_2018_01461 crossref_primary_10_1080_13537121_2017_1333725 crossref_primary_10_1080_02684527_2019_1569343 crossref_primary_10_1080_24733938_2022_2100461 crossref_primary_10_1177_1541931215591249 crossref_primary_10_1080_02684527_2024_2304934 crossref_primary_10_1016_j_tics_2021_06_004 crossref_primary_10_1080_02684527_2021_1999621 crossref_primary_10_1111_risa_12992 crossref_primary_10_3758_s13428_021_01604_7 crossref_primary_10_1177_10464964251361563 crossref_primary_10_2139_ssrn_2819820 crossref_primary_10_1080_13546783_2023_2259548 crossref_primary_10_1016_j_tsc_2020_100659 crossref_primary_10_1080_0163853X_2024_2355059 crossref_primary_10_1111_ssqu_12601 crossref_primary_10_1080_13575279_2021_2001434 crossref_primary_10_1126_science_aal3147 crossref_primary_10_1080_23311908_2022_2047300 crossref_primary_10_3389_fpsyg_2019_02749 crossref_primary_10_1177_2053168016676705 |
| ContentType | Journal Article |
| Copyright | PsycINFO Database Record (c) 2015 APA, all rights reserved. |
| Copyright_xml | – notice: PsycINFO Database Record (c) 2015 APA, all rights reserved. |
| DBID | CGR CUY CVF ECM EIF NPM 7X8 |
| DOI | 10.1037/xap0000040 |
| DatabaseName | Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed MEDLINE - Academic |
| DatabaseTitle | MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) MEDLINE - Academic |
| DatabaseTitleList | MEDLINE MEDLINE - Academic |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | no_fulltext_linktorsrc |
| Discipline | Psychology |
| EISSN | 1939-2192 |
| ExternalDocumentID | 25581088 |
| Genre | Research Support, U.S. Gov't, Non-P.H.S Journal Article |
| GroupedDBID | --- -DZ 0R~ 29K 354 5GY 5VS 7RZ 8VB 9M8 ABIVO ABNCP ABVOZ ACGFO ACPQG AEHFB AEMOZ AETEA AHQJS AI. AKVCP ALMA_UNASSIGNED_HOLDINGS AWKKM AZXWR CGNQK CGR CS3 CUY CVF D-I EBR EBU ECM EIF EMK EPA F5P FTD HVGLF HZ~ ISO K1G LW5 NPM O9- OPA OVD P2P PHGZT QWB ROL SES SPA TEORI TH9 TN5 UHS UPT VH1 WH7 YQT ZL0 ZPI 3KI 7X8 PUEGO |
| ID | FETCH-LOGICAL-a419t-70102a1f7b80ef5b069691a7d8ae9268f25813a4a5573c99cfc94f20b17e739f2 |
| IEDL.DBID | 7X8 |
| ISICitedReferencesCount | 135 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000350985700001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 1939-2192 |
| IngestDate | Sun Sep 28 02:39:10 EDT 2025 Thu Apr 03 07:01:51 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 1 |
| Language | English |
| License | PsycINFO Database Record (c) 2015 APA, all rights reserved. |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a419t-70102a1f7b80ef5b069691a7d8ae9268f25813a4a5573c99cfc94f20b17e739f2 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 |
| PMID | 25581088 |
| PQID | 1662433180 |
| PQPubID | 23479 |
| ParticipantIDs | proquest_miscellaneous_1662433180 pubmed_primary_25581088 |
| PublicationCentury | 2000 |
| PublicationDate | 2015-03-01 |
| PublicationDateYYYYMMDD | 2015-03-01 |
| PublicationDate_xml | – month: 03 year: 2015 text: 2015-03-01 day: 01 |
| PublicationDecade | 2010 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | Journal of experimental psychology. Applied |
| PublicationTitleAlternate | J Exp Psychol Appl |
| PublicationYear | 2015 |
| SSID | ssj0008994 |
| Score | 2.491046 |
| Snippet | This article extends psychological methods and concepts into a domain that is as profoundly consequential as it is poorly understood: intelligence analysis. We... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| StartPage | 1 |
| SubjectTerms | Adult Female Forecasting Humans Intelligence Judgment Male Models, Statistical Politics Probability Psychological Techniques |
| Title | The psychology of intelligence analysis: drivers of prediction accuracy in world politics |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/25581088 https://www.proquest.com/docview/1662433180 |
| Volume | 21 |
| WOSCitedRecordID | wos000350985700001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV07T8MwELaAMnTh_SgvGYnVahzHsc2CEKJigKoDSGWKbCcndUlD0iL677GTtGVBQmLJYMVSdL77_Nl3-Q6hG7-HG8aAcBZbEinGiWEu8MDYiApKdQa1zuyzGA7leKxG7YVb1ZZVLjGxBup0av0deZ_Gcej_7pHBXfFBfNcon11tW2hsog5zVMaXdInxWi3cnSXarLIiLjLDpTwpE_0vXTT8OPidWtZbzGD3vx-3h3ZaconvG2_YRxtZfoC6K4xbHKJ35xa4WA3gKeDJD1FOrFuNkluclnXFhn-jKH02x68g1tbOS20XbhauxVZx0RTQVUfobfD4-vBE2u4KREdUzYjwanKagjAyyICbIFaxolqkUmcqjCWEXFKmI825YFYpC1ZFEAaGikwwBeEx2sqneXaKMItZAA4ZGABEkoMx1KRKpEIYDiB5D10vzZY47_UpCZ1n03mVrA3XQyeN7ZOikdlI3GFHUgeCZ3-YfY66jsnwpjjsAnXAxW52ibbt52xSlVe1W7jncPTyDQsmwxw |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=The+psychology+of+intelligence+analysis%3A+drivers+of+prediction+accuracy+in+world+politics&rft.jtitle=Journal+of+experimental+psychology.+Applied&rft.au=Mellers%2C+Barbara&rft.au=Stone%2C+Eric&rft.au=Atanasov%2C+Pavel&rft.au=Rohrbaugh%2C+Nick&rft.date=2015-03-01&rft.eissn=1939-2192&rft.volume=21&rft.issue=1&rft.spage=1&rft_id=info:doi/10.1037%2Fxap0000040&rft_id=info%3Apmid%2F25581088&rft_id=info%3Apmid%2F25581088&rft.externalDocID=25581088 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=1939-2192&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=1939-2192&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=1939-2192&client=summon |