Reinforcement Learning in Patients With Mood and Anxiety Disorders vs Control Individuals: A Systematic Review and Meta-analysis
Computational psychiatry studies have investigated how reinforcement learning may be different in individuals with mood and anxiety disorders compared with control individuals, but results are inconsistent. To assess whether there are consistent differences in reinforcement-learning parameters betwe...
Uloženo v:
| Vydáno v: | JAMA psychiatry (Chicago, Ill.) Ročník 79; číslo 4; s. 313 |
|---|---|
| Hlavní autoři: | , |
| Médium: | Journal Article |
| Jazyk: | angličtina |
| Vydáno: |
United States
01.04.2022
|
| Témata: | |
| ISSN: | 2168-6238, 2168-6238 |
| On-line přístup: | Zjistit podrobnosti o přístupu |
| Tagy: |
Přidat tag
Žádné tagy, Buďte první, kdo vytvoří štítek k tomuto záznamu!
|
| Abstract | Computational psychiatry studies have investigated how reinforcement learning may be different in individuals with mood and anxiety disorders compared with control individuals, but results are inconsistent.
To assess whether there are consistent differences in reinforcement-learning parameters between patients with depression or anxiety and control individuals.
Web of Knowledge, PubMed, Embase, and Google Scholar searches were performed between November 15, 2019, and December 6, 2019, and repeated on December 3, 2020, and February 23, 2021, with keywords (reinforcement learning) AND (computational OR model) AND (depression OR anxiety OR mood).
Studies were included if they fit reinforcement-learning models to human choice data from a cognitive task with rewards or punishments, had a case-control design including participants with mood and/or anxiety disorders and healthy control individuals, and included sufficient information about all parameters in the models.
Articles were assessed for inclusion according to MOOSE guidelines. Participant-level parameters were extracted from included articles, and a conventional meta-analysis was performed using a random-effects model. Subsequently, these parameters were used to simulate choice performance for each participant on benchmarking tasks in a simulation meta-analysis. Models were fitted, parameters were extracted using bayesian model averaging, and differences between patients and control individuals were examined. Overall effect sizes across analytic strategies were inspected.
The primary outcomes were estimated reinforcement-learning parameters (learning rate, inverse temperature, reward learning rate, and punishment learning rate).
A total of 27 articles were included (3085 participants, 1242 of whom had depression and/or anxiety). In the conventional meta-analysis, patients showed lower inverse temperature than control individuals (standardized mean difference [SMD], -0.215; 95% CI, -0.354 to -0.077), although no parameters were common across all studies, limiting the ability to infer differences. In the simulation meta-analysis, patients showed greater punishment learning rates (SMD, 0.107; 95% CI, 0.107 to 0.108) and slightly lower reward learning rates (SMD, -0.021; 95% CI, -0.022 to -0.020) relative to control individuals. The simulation meta-analysis showed no meaningful difference in inverse temperature between patients and control individuals (SMD, 0.003; 95% CI, 0.002 to 0.004).
The simulation meta-analytic approach introduced in this article for inferring meta-group differences from heterogeneous computational psychiatry studies indicated elevated punishment learning rates in patients compared with control individuals. This difference may promote and uphold negative affective bias symptoms and hence constitute a potential mechanistic treatment target for mood and anxiety disorders. |
|---|---|
| AbstractList | Computational psychiatry studies have investigated how reinforcement learning may be different in individuals with mood and anxiety disorders compared with control individuals, but results are inconsistent.IMPORTANCEComputational psychiatry studies have investigated how reinforcement learning may be different in individuals with mood and anxiety disorders compared with control individuals, but results are inconsistent.To assess whether there are consistent differences in reinforcement-learning parameters between patients with depression or anxiety and control individuals.OBJECTIVETo assess whether there are consistent differences in reinforcement-learning parameters between patients with depression or anxiety and control individuals.Web of Knowledge, PubMed, Embase, and Google Scholar searches were performed between November 15, 2019, and December 6, 2019, and repeated on December 3, 2020, and February 23, 2021, with keywords (reinforcement learning) AND (computational OR model) AND (depression OR anxiety OR mood).DATA SOURCESWeb of Knowledge, PubMed, Embase, and Google Scholar searches were performed between November 15, 2019, and December 6, 2019, and repeated on December 3, 2020, and February 23, 2021, with keywords (reinforcement learning) AND (computational OR model) AND (depression OR anxiety OR mood).Studies were included if they fit reinforcement-learning models to human choice data from a cognitive task with rewards or punishments, had a case-control design including participants with mood and/or anxiety disorders and healthy control individuals, and included sufficient information about all parameters in the models.STUDY SELECTIONStudies were included if they fit reinforcement-learning models to human choice data from a cognitive task with rewards or punishments, had a case-control design including participants with mood and/or anxiety disorders and healthy control individuals, and included sufficient information about all parameters in the models.Articles were assessed for inclusion according to MOOSE guidelines. Participant-level parameters were extracted from included articles, and a conventional meta-analysis was performed using a random-effects model. Subsequently, these parameters were used to simulate choice performance for each participant on benchmarking tasks in a simulation meta-analysis. Models were fitted, parameters were extracted using bayesian model averaging, and differences between patients and control individuals were examined. Overall effect sizes across analytic strategies were inspected.DATA EXTRACTION AND SYNTHESISArticles were assessed for inclusion according to MOOSE guidelines. Participant-level parameters were extracted from included articles, and a conventional meta-analysis was performed using a random-effects model. Subsequently, these parameters were used to simulate choice performance for each participant on benchmarking tasks in a simulation meta-analysis. Models were fitted, parameters were extracted using bayesian model averaging, and differences between patients and control individuals were examined. Overall effect sizes across analytic strategies were inspected.The primary outcomes were estimated reinforcement-learning parameters (learning rate, inverse temperature, reward learning rate, and punishment learning rate).MAIN OUTCOMES AND MEASURESThe primary outcomes were estimated reinforcement-learning parameters (learning rate, inverse temperature, reward learning rate, and punishment learning rate).A total of 27 articles were included (3085 participants, 1242 of whom had depression and/or anxiety). In the conventional meta-analysis, patients showed lower inverse temperature than control individuals (standardized mean difference [SMD], -0.215; 95% CI, -0.354 to -0.077), although no parameters were common across all studies, limiting the ability to infer differences. In the simulation meta-analysis, patients showed greater punishment learning rates (SMD, 0.107; 95% CI, 0.107 to 0.108) and slightly lower reward learning rates (SMD, -0.021; 95% CI, -0.022 to -0.020) relative to control individuals. The simulation meta-analysis showed no meaningful difference in inverse temperature between patients and control individuals (SMD, 0.003; 95% CI, 0.002 to 0.004).RESULTSA total of 27 articles were included (3085 participants, 1242 of whom had depression and/or anxiety). In the conventional meta-analysis, patients showed lower inverse temperature than control individuals (standardized mean difference [SMD], -0.215; 95% CI, -0.354 to -0.077), although no parameters were common across all studies, limiting the ability to infer differences. In the simulation meta-analysis, patients showed greater punishment learning rates (SMD, 0.107; 95% CI, 0.107 to 0.108) and slightly lower reward learning rates (SMD, -0.021; 95% CI, -0.022 to -0.020) relative to control individuals. The simulation meta-analysis showed no meaningful difference in inverse temperature between patients and control individuals (SMD, 0.003; 95% CI, 0.002 to 0.004).The simulation meta-analytic approach introduced in this article for inferring meta-group differences from heterogeneous computational psychiatry studies indicated elevated punishment learning rates in patients compared with control individuals. This difference may promote and uphold negative affective bias symptoms and hence constitute a potential mechanistic treatment target for mood and anxiety disorders.CONCLUSIONS AND RELEVANCEThe simulation meta-analytic approach introduced in this article for inferring meta-group differences from heterogeneous computational psychiatry studies indicated elevated punishment learning rates in patients compared with control individuals. This difference may promote and uphold negative affective bias symptoms and hence constitute a potential mechanistic treatment target for mood and anxiety disorders. Computational psychiatry studies have investigated how reinforcement learning may be different in individuals with mood and anxiety disorders compared with control individuals, but results are inconsistent. To assess whether there are consistent differences in reinforcement-learning parameters between patients with depression or anxiety and control individuals. Web of Knowledge, PubMed, Embase, and Google Scholar searches were performed between November 15, 2019, and December 6, 2019, and repeated on December 3, 2020, and February 23, 2021, with keywords (reinforcement learning) AND (computational OR model) AND (depression OR anxiety OR mood). Studies were included if they fit reinforcement-learning models to human choice data from a cognitive task with rewards or punishments, had a case-control design including participants with mood and/or anxiety disorders and healthy control individuals, and included sufficient information about all parameters in the models. Articles were assessed for inclusion according to MOOSE guidelines. Participant-level parameters were extracted from included articles, and a conventional meta-analysis was performed using a random-effects model. Subsequently, these parameters were used to simulate choice performance for each participant on benchmarking tasks in a simulation meta-analysis. Models were fitted, parameters were extracted using bayesian model averaging, and differences between patients and control individuals were examined. Overall effect sizes across analytic strategies were inspected. The primary outcomes were estimated reinforcement-learning parameters (learning rate, inverse temperature, reward learning rate, and punishment learning rate). A total of 27 articles were included (3085 participants, 1242 of whom had depression and/or anxiety). In the conventional meta-analysis, patients showed lower inverse temperature than control individuals (standardized mean difference [SMD], -0.215; 95% CI, -0.354 to -0.077), although no parameters were common across all studies, limiting the ability to infer differences. In the simulation meta-analysis, patients showed greater punishment learning rates (SMD, 0.107; 95% CI, 0.107 to 0.108) and slightly lower reward learning rates (SMD, -0.021; 95% CI, -0.022 to -0.020) relative to control individuals. The simulation meta-analysis showed no meaningful difference in inverse temperature between patients and control individuals (SMD, 0.003; 95% CI, 0.002 to 0.004). The simulation meta-analytic approach introduced in this article for inferring meta-group differences from heterogeneous computational psychiatry studies indicated elevated punishment learning rates in patients compared with control individuals. This difference may promote and uphold negative affective bias symptoms and hence constitute a potential mechanistic treatment target for mood and anxiety disorders. |
| Author | Robinson, Oliver J Pike, Alexandra C |
| Author_xml | – sequence: 1 givenname: Alexandra C surname: Pike fullname: Pike, Alexandra C organization: Anxiety Lab, Neuroscience and Mental Health Group, Institute of Cognitive Neuroscience, University College London, London, United Kingdom – sequence: 2 givenname: Oliver J surname: Robinson fullname: Robinson, Oliver J organization: Research Department of Clinical, Educational and Health Psychology, University College London, London, United Kingdom |
| BackLink | https://www.ncbi.nlm.nih.gov/pubmed/35234834$$D View this record in MEDLINE/PubMed |
| BookMark | eNpNkE1PAjEQhhuDEUT-gunRy2J3WmDxRvCLBKJBjUcydAcp2W2xLeje_OluFBPnMpM3T55k3lPWsM4SYzwV3VSI9HKDJW5DpdcGo6-6IAC6QvTSI9aCtJ8lfZBZ49_dZJ0QNqKeTAglsxPWlD2QKpOqxb7mZOzKeU0l2cinhN4a-8aN5Y8YTZ0F_mrims-cyznanI_sp6FY8WsTnM_JB74PfOxs9K7gE5ubvcl3WIQrPuJPVYhU1h7N57Q39PFjmFHEBC0WVTDhjB2vapo6h91mL7c3z-P7ZPpwNxmPpgmqdBATSQp1PwOEJWZSDnUKGlCC1LlYShwiDpY1CXm6Urr-bkBay9UQtAClFAK02cWvd-vd-45CXJQmaCoKtOR2YQH9uhTVk5DV6PkB3S1Lyhdbb0r01eKvNfgGczN3Ew |
| CitedBy_id | crossref_primary_10_1007_s12144_025_07810_2 crossref_primary_10_1038_s41380_025_02980_8 crossref_primary_10_1111_tops_12701 crossref_primary_10_2147_IJGM_S453903 crossref_primary_10_1017_S0033291724000837 crossref_primary_10_3758_s13415_023_01137_w crossref_primary_10_1016_j_tics_2022_12_004 crossref_primary_10_1038_s41467_024_48548_y crossref_primary_10_7554_eLife_93887_3 crossref_primary_10_1038_s44220_025_00427_1 crossref_primary_10_1038_s44220_025_00465_9 crossref_primary_10_59324_stss_2025_2_4__10 crossref_primary_10_1038_s41380_024_02459_y crossref_primary_10_1016_j_brs_2025_01_022 crossref_primary_10_5498_wjp_v15_i8_106025 crossref_primary_10_1016_j_janxdis_2022_102634 crossref_primary_10_1016_j_cell_2024_09_028 crossref_primary_10_1016_j_biopsych_2023_02_004 crossref_primary_10_1016_j_brat_2024_104664 crossref_primary_10_3389_fnins_2022_889440 crossref_primary_10_1016_j_neuroimage_2023_120029 crossref_primary_10_1017_S0033291722001593 crossref_primary_10_1371_journal_pcbi_1013445 crossref_primary_10_7554_eLife_87720_4 crossref_primary_10_1016_j_biopsych_2025_05_011 crossref_primary_10_1016_j_neubiorev_2023_105137 crossref_primary_10_1017_S0033291723002519 crossref_primary_10_1038_s41386_024_01946_8 crossref_primary_10_1038_s42003_023_04544_4 crossref_primary_10_1016_j_tics_2024_01_006 crossref_primary_10_1017_S0033291723001587 crossref_primary_10_1002_mhs2_49 crossref_primary_10_1038_s41598_024_64240_z crossref_primary_10_1001_jamapsychiatry_2025_0839 crossref_primary_10_1080_01621459_2023_2261184 crossref_primary_10_3758_s13415_023_01080_w crossref_primary_10_1016_j_tics_2023_04_002 crossref_primary_10_1016_j_neubiorev_2022_104959 crossref_primary_10_3758_s13423_024_02490_8 crossref_primary_10_1111_insr_12583 crossref_primary_10_1017_S0033291723002520 crossref_primary_10_1038_s41598_023_42628_7 crossref_primary_10_1016_j_neubiorev_2022_105008 crossref_primary_10_1016_j_neubiorev_2022_104995 crossref_primary_10_3389_fpsyt_2024_1492332 crossref_primary_10_1016_j_biopsych_2024_01_020 crossref_primary_10_1016_j_bpsc_2024_01_011 crossref_primary_10_1016_j_nlm_2025_108092 crossref_primary_10_1038_s41467_025_63281_w crossref_primary_10_1016_j_neubiorev_2023_105123 crossref_primary_10_1016_j_neubiorev_2023_105087 crossref_primary_10_1038_s41598_022_26055_8 crossref_primary_10_1038_s41598_023_45179_z crossref_primary_10_1016_j_jad_2023_02_120 crossref_primary_10_1177_21677026231213368 crossref_primary_10_3390_brainsci14121278 crossref_primary_10_3390_fi16090343 crossref_primary_10_1016_j_biopsych_2022_08_016 crossref_primary_10_1016_j_jpsychires_2023_01_001 crossref_primary_10_1371_journal_pcbi_1010751 crossref_primary_10_7554_eLife_87720 crossref_primary_10_1038_s41398_025_03236_3 crossref_primary_10_1186_s13229_024_00610_8 crossref_primary_10_1016_j_tics_2023_08_016 crossref_primary_10_1038_s41598_025_05725_3 crossref_primary_10_1177_09637214231194962 crossref_primary_10_3758_s13415_025_01331_y crossref_primary_10_1016_j_bpsc_2023_01_001 crossref_primary_10_1007_s42113_024_00229_1 crossref_primary_10_7554_eLife_93887 crossref_primary_10_3389_fncom_2024_1466364 crossref_primary_10_1016_j_biopsych_2024_12_012 crossref_primary_10_1038_s41398_024_03204_3 crossref_primary_10_1038_s41598_024_72749_6 |
| ContentType | Journal Article |
| DBID | CGR CUY CVF ECM EIF NPM 7X8 |
| DOI | 10.1001/jamapsychiatry.2022.0051 |
| DatabaseName | Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed MEDLINE - Academic |
| DatabaseTitle | MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) MEDLINE - Academic |
| DatabaseTitleList | MEDLINE - Academic MEDLINE |
| Database_xml | – sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database |
| DeliveryMethod | no_fulltext_linktorsrc |
| Discipline | Medicine |
| EISSN | 2168-6238 |
| ExternalDocumentID | 35234834 |
| Genre | Meta-Analysis Research Support, Non-U.S. Gov't Systematic Review Journal Article |
| GrantInformation_xml | – fundername: Medical Research Council grantid: MR/R020817/1 |
| GroupedDBID | -DZ 0R~ 4.4 5RS 9M8 AAGZG ABIVO ABJNI ABPMR ACDNT ACGFS ACHQT ACNCT ADBBV ADHGD AENEX AFCHL AHMBA ALMA_UNASSIGNED_HOLDINGS AMJDE ANMPU BRYMA C45 CGR CUY CVF EBD EBS ECM EIF EJD EMOBN EX3 HF~ KOO M5~ NPM OB2 OBH OCB OFXIZ OGEVE OHH OMH OVD PQQKQ RAJ SV3 TEORI WH7 WOW XJT 7X8 |
| ID | FETCH-LOGICAL-a417t-3e4ac682a2ba8339c12c2a323cd0b3a9aa7ba412d1f4c5237ecc3f92c02444a22 |
| IEDL.DBID | 7X8 |
| ISICitedReferencesCount | 80 |
| ISICitedReferencesURI | http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=000764268300005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| ISSN | 2168-6238 |
| IngestDate | Sun Nov 09 13:57:46 EST 2025 Mon Jul 21 05:46:56 EDT 2025 |
| IsDoiOpenAccess | false |
| IsOpenAccess | true |
| IsPeerReviewed | true |
| IsScholarly | true |
| Issue | 4 |
| Language | English |
| LinkModel | DirectLink |
| MergedId | FETCHMERGED-LOGICAL-a417t-3e4ac682a2ba8339c12c2a323cd0b3a9aa7ba412d1f4c5237ecc3f92c02444a22 |
| Notes | ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 23 ObjectType-Undefined-3 |
| OpenAccessLink | https://pubmed.ncbi.nlm.nih.gov/PMC8892374 |
| PMID | 35234834 |
| PQID | 2635245328 |
| PQPubID | 23479 |
| ParticipantIDs | proquest_miscellaneous_2635245328 pubmed_primary_35234834 |
| PublicationCentury | 2000 |
| PublicationDate | 2022-04-01 |
| PublicationDateYYYYMMDD | 2022-04-01 |
| PublicationDate_xml | – month: 04 year: 2022 text: 2022-04-01 day: 01 |
| PublicationDecade | 2020 |
| PublicationPlace | United States |
| PublicationPlace_xml | – name: United States |
| PublicationTitle | JAMA psychiatry (Chicago, Ill.) |
| PublicationTitleAlternate | JAMA Psychiatry |
| PublicationYear | 2022 |
| SSID | ssj0000800438 |
| Score | 2.6295967 |
| SecondaryResourceType | review_article |
| Snippet | Computational psychiatry studies have investigated how reinforcement learning may be different in individuals with mood and anxiety disorders compared with... |
| SourceID | proquest pubmed |
| SourceType | Aggregation Database Index Database |
| StartPage | 313 |
| SubjectTerms | Affect Anxiety - therapy Anxiety Disorders - diagnosis Anxiety Disorders - therapy Bayes Theorem Humans Reward |
| Title | Reinforcement Learning in Patients With Mood and Anxiety Disorders vs Control Individuals: A Systematic Review and Meta-analysis |
| URI | https://www.ncbi.nlm.nih.gov/pubmed/35234834 https://www.proquest.com/docview/2635245328 |
| Volume | 79 |
| WOSCitedRecordID | wos000764268300005&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D |
| hasFullText | |
| inHoldings | 1 |
| isFullTextHit | |
| isPrint | |
| link | http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwpV1LS8NAEF7Uinjx_agvVvC62Oxuko0XKcWiYErxgb2Vze5Ge0mqicWjP93ZZNOeBMFLLsmEsDPM88s3CF1oo2VKjSHcpFCgmMgnUmiPJImiiemkAas1fR8OBmI0ioau4VY4WGXjEytHrXNle-SXljSFcp9RcT19J3ZrlJ2uuhUay6jFIJWxVh2OxLzHYrMhXi2zpl4gCER64cA8DfHQAlIMpSK13J2-93uuWcWc_uZ_v3YLbbhsE3dr89hGSybbQWuxm6fvou8HUzGnqqpJiB3Z6iueZHhYE64W-GVSvuE4zzWWmcbd7MuiPHHD2lngWYF7Nd4d383_7iqucBc_zlmicT2CqN4Qm1IS6bhQ9tBz_-apd0vcTgYiuReWhBkuVSCopIkUjEXKo4pKRpnSnYTJSMowgSep9lKuoMgNwURYGlEFuQDnktJ9tJLlmTlEOArTUERQwQUaQiTIcQ3JoQ-C4EW58trovDnbMdi8HWTIzOSfxXhxum10UCtoPK3JOcZwh9kG6dEfpI_RutV6DcQ5Qa0UjsecolU1KyfFx1llTHAdDOMfqvLWXQ |
| linkProvider | ProQuest |
| openUrl | ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Reinforcement+Learning+in+Patients+With+Mood+and+Anxiety+Disorders+vs+Control+Individuals%3A+A+Systematic+Review+and+Meta-analysis&rft.jtitle=JAMA+psychiatry+%28Chicago%2C+Ill.%29&rft.au=Pike%2C+Alexandra+C&rft.au=Robinson%2C+Oliver+J&rft.date=2022-04-01&rft.eissn=2168-6238&rft.volume=79&rft.issue=4&rft.spage=313&rft_id=info:doi/10.1001%2Fjamapsychiatry.2022.0051&rft_id=info%3Apmid%2F35234834&rft_id=info%3Apmid%2F35234834&rft.externalDocID=35234834 |
| thumbnail_l | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=2168-6238&client=summon |
| thumbnail_m | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=2168-6238&client=summon |
| thumbnail_s | http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=2168-6238&client=summon |