Curiosity-driven exploration: foundations in neuroscience and computational modeling

Exploratory behaviors are central to the study of curiosity, since they allow us to infer an otherwise hidden state of being curious from quantitative, experimental observations.Reactive orienting behaviors reflect a fundamental form of curiosity and are driven by the immediate novelty and/or surpri...

Full description

Saved in:

Bibliographic Details
Published in:	Trends in neurosciences (Regular ed.) Vol. 46; no. 12; pp. 1054 - 1066
Main Authors:	Modirshanechi, Alireza, Kondrakiewicz, Kacper, Gerstner, Wulfram, Haesler, Sebastian
Format:	Journal Article
Language:	English
Published:	England Elsevier Ltd 01.12.2023
Subjects:	Animals Computer Simulation Exploratory Behavior - physiology Humans information Motivation neural circuits neuromodulatory systems Neurosciences novelty orienting reinforcement learning surprise surprise orienting novelty neural circuits information reinforcement learning neuromodulatory systems
ISSN:	0166-2236, 1878-108X, 1878-108X
Online Access:	Get full text
Tags:	Add Tag No Tags, Be the first to tag this record!

Abstract	Exploratory behaviors are central to the study of curiosity, since they allow us to infer an otherwise hidden state of being curious from quantitative, experimental observations.Reactive orienting behaviors reflect a fundamental form of curiosity and are driven by the immediate novelty and/or surprise of stimuli.Operant tasks probe curiosity by offering experimental subjects explicit action choices to collect information, in a manner that is either neutral or costly/useful with regards to overall gains/losses.Recently developed computational models of curiosity, based on intrinsically motivated reinforcement learning, describe different expressions of curiosity in a unified theoretical framework. Curiosity refers to the intrinsic desire of humans and animals to explore the unknown, even when there is no apparent reason to do so. Thus far, no single, widely accepted definition or framework for curiosity has emerged, but there is growing consensus that curious behavior is not goal-directed but related to seeking or reacting to information. In this review, we take a phenomenological approach and group behavioral and neurophysiological studies which meet these criteria into three categories according to the type of information seeking observed. We then review recent computational models of curiosity from the field of machine learning and discuss how they enable integrating different types of information seeking into one theoretical framework. Combinations of behavioral and neurophysiological studies along with computational modeling will be instrumental in demystifying the notion of curiosity. Curiosity refers to the intrinsic desire of humans and animals to explore the unknown, even when there is no apparent reason to do so. Thus far, no single, widely accepted definition or framework for curiosity has emerged, but there is growing consensus that curious behavior is not goal-directed but related to seeking or reacting to information. In this review, we take a phenomenological approach and group behavioral and neurophysiological studies which meet these criteria into three categories according to the type of information seeking observed. We then review recent computational models of curiosity from the field of machine learning and discuss how they enable integrating different types of information seeking into one theoretical framework. Combinations of behavioral and neurophysiological studies along with computational modeling will be instrumental in demystifying the notion of curiosity.
AbstractList	Curiosity refers to the intrinsic desire of humans and animals to explore the unknown, even when there is no apparent reason to do so. Thus far, no single, widely accepted definition or framework for curiosity has emerged, but there is growing consensus that curious behavior is not goal-directed but related to seeking or reacting to information. In this review, we take a phenomenological approach and group behavioral and neurophysiological studies which meet these criteria into three categories according to the type of information seeking observed. We then review recent computational models of curiosity from the field of machine learning and discuss how they enable integrating different types of information seeking into one theoretical framework. Combinations of behavioral and neurophysiological studies along with computational modeling will be instrumental in demystifying the notion of curiosity. Exploratory behaviors are central to the study of curiosity, since they allow us to infer an otherwise hidden state of being curious from quantitative, experimental observations.Reactive orienting behaviors reflect a fundamental form of curiosity and are driven by the immediate novelty and/or surprise of stimuli.Operant tasks probe curiosity by offering experimental subjects explicit action choices to collect information, in a manner that is either neutral or costly/useful with regards to overall gains/losses.Recently developed computational models of curiosity, based on intrinsically motivated reinforcement learning, describe different expressions of curiosity in a unified theoretical framework. Curiosity refers to the intrinsic desire of humans and animals to explore the unknown, even when there is no apparent reason to do so. Thus far, no single, widely accepted definition or framework for curiosity has emerged, but there is growing consensus that curious behavior is not goal-directed but related to seeking or reacting to information. In this review, we take a phenomenological approach and group behavioral and neurophysiological studies which meet these criteria into three categories according to the type of information seeking observed. We then review recent computational models of curiosity from the field of machine learning and discuss how they enable integrating different types of information seeking into one theoretical framework. Combinations of behavioral and neurophysiological studies along with computational modeling will be instrumental in demystifying the notion of curiosity. Curiosity refers to the intrinsic desire of humans and animals to explore the unknown, even when there is no apparent reason to do so. Thus far, no single, widely accepted definition or framework for curiosity has emerged, but there is growing consensus that curious behavior is not goal-directed but related to seeking or reacting to information. In this review, we take a phenomenological approach and group behavioral and neurophysiological studies which meet these criteria into three categories according to the type of information seeking observed. We then review recent computational models of curiosity from the field of machine learning and discuss how they enable integrating different types of information seeking into one theoretical framework. Combinations of behavioral and neurophysiological studies along with computational modeling will be instrumental in demystifying the notion of curiosity. Curiosity refers to the intrinsic desire of humans and animals to explore the unknown, even when there is no apparent reason to do so. Thus far, no single, widely accepted definition or framework for curiosity has emerged, but there is growing consensus that curious behavior is not goal-directed but related to seeking or reacting to information. In this review, we take a phenomenological approach and group behavioral and neurophysiological studies which meet these criteria into three categories according to the type of information seeking observed. We then review recent computational models of curiosity from the field of machine learning and discuss how they enable integrating different types of information seeking into one theoretical framework. Combinations of behavioral and neurophysiological studies along with computational modeling will be instrumental in demystifying the notion of curiosity.Curiosity refers to the intrinsic desire of humans and animals to explore the unknown, even when there is no apparent reason to do so. Thus far, no single, widely accepted definition or framework for curiosity has emerged, but there is growing consensus that curious behavior is not goal-directed but related to seeking or reacting to information. In this review, we take a phenomenological approach and group behavioral and neurophysiological studies which meet these criteria into three categories according to the type of information seeking observed. We then review recent computational models of curiosity from the field of machine learning and discuss how they enable integrating different types of information seeking into one theoretical framework. Combinations of behavioral and neurophysiological studies along with computational modeling will be instrumental in demystifying the notion of curiosity.
Author	Haesler, Sebastian Kondrakiewicz, Kacper Gerstner, Wulfram Modirshanechi, Alireza
Author_xml	– sequence: 1 givenname: Alireza surname: Modirshanechi fullname: Modirshanechi, Alireza organization: École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland – sequence: 2 givenname: Kacper surname: Kondrakiewicz fullname: Kondrakiewicz, Kacper organization: Neuroelectronics Research Flanders (NERF), Leuven, Belgium – sequence: 3 givenname: Wulfram surname: Gerstner fullname: Gerstner, Wulfram email: wulfram.gerstner@epfl.ch organization: École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland – sequence: 4 givenname: Sebastian orcidid: 0000-0003-4924-7381 surname: Haesler fullname: Haesler, Sebastian email: sebastian.haesler@nerf.be organization: Neuroelectronics Research Flanders (NERF), Leuven, Belgium
BackLink	https://www.ncbi.nlm.nih.gov/pubmed/37925342$$D View this record in MEDLINE/PubMed
BookMark	eNp9kMtOAyEUQImpsQ_9ARdmlm6mAvOAdmcaX0kTNzVxRyjcMdQZqDDT2L-XPnThoivIzTmEe4aoZ50FhK4JHhNMyrvVuDU2jCmmWRyMMaZnaEA44ynB_L2HBhEqU0qzso-GIawwJjkn-QXqZ2xCiyynA7SYdd64YNptqr3ZgE3ge107L1vj7DSpXGf1_h4SYxMLnXdBGbAKEml1olyz7to9IOukcRpqYz8u0Xkl6wBXx3OE3h4fFrPndP769DK7n6cqL0ib5sAnMqcaM2CFZJIALRjhS6wyUlEptY6AYvGzuiiWPOeYEcKV4hXVZYmrbIRuD--uvfvqILSiMUFBXUsLrguCcl5mdEJKFtGbI9otG9Bi7U0j_Vb8logAPQAqbhg8VH8IwWKXW6zELrfY5d7NYu4o8X-SMocarZemPq1ODyrEQBsDXqiYzihZf8JWaGdOyT_IBpuT
CitedBy_id	crossref_primary_10_1016_j_watres_2025_123842 crossref_primary_10_1017_S0140525X24001353 crossref_primary_10_1038_s41598_024_66316_2 crossref_primary_10_1038_s41583_025_00968_5 crossref_primary_10_1038_s41598_025_14479_x crossref_primary_10_1109_TCDS_2024_3454779 crossref_primary_10_1111_nyas_15302 crossref_primary_10_1038_s42003_025_07486_1 crossref_primary_10_3758_s13420_024_00647_y crossref_primary_10_1038_s44159_024_00346_5 crossref_primary_10_1097_j_pain_0000000000003222 crossref_primary_10_1007_s42113_025_00256_6 crossref_primary_10_1017_S0140525X24000505 crossref_primary_10_1017_S0140525X23003424 crossref_primary_10_1080_29973368_2025_2547280 crossref_primary_10_1073_pnas_2415236121 crossref_primary_10_1016_j_celrep_2025_115239 crossref_primary_10_3389_feduc_2025_1502203 crossref_primary_10_1007_s12124_024_09856_6 crossref_primary_10_1007_s40806_024_00400_8
Cites_doi	10.1037/a0038199 10.1037/h0054833 10.1016/j.cub.2018.03.024 10.7554/eLife.59907 10.1016/S0028-3908(98)00030-6 10.1038/nn.3185 10.1016/j.jmp.2022.102712 10.1371/journal.pbio.3000744 10.1126/science.abe9681 10.1016/j.cub.2017.04.027 10.1101/lm.053392.121 10.1037/0033-295X.112.4.979 10.1016/j.neuron.2020.10.029 10.1523/JNEUROSCI.18-06-02226.1998 10.1111/brv.12560 10.1037/rev0000349 10.1163/156853973X00247 10.1037/0033-2909.116.1.75 10.1016/j.brainres.2018.05.008 10.1016/j.neuron.2012.09.037 10.1016/j.neubiorev.2020.10.022 10.1016/j.neuroimage.2007.06.038 10.1037/h0071833 10.1152/jn.1983.49.5.1285 10.1016/j.neuroscience.2017.08.034 10.1038/s41467-022-33585-2 10.7554/eLife.18044 10.1037/h0041823 10.1038/s41467-021-26196-w 10.1109/TAMD.2010.2056368 10.1016/j.visres.2015.10.009 10.1111/tops.12138 10.1016/j.neuron.2009.06.009 10.1016/j.neuroimage.2019.04.028 10.1038/s41562-018-0467-4 10.1016/j.neuron.2008.04.027 10.1016/j.neuron.2006.02.015 10.1152/jn.91048.2008 10.1016/j.conb.2021.10.005 10.1007/s42113-021-00112-3 10.1016/j.neubiorev.2015.05.002 10.1126/science.275.5306.1593 10.1038/nn.2902 10.1126/science.146.3644.668 10.1111/j.1469-8986.2010.01057.x 10.1038/nature25457 10.1038/s41562-020-0848-3 10.1016/j.jcss.2007.08.009 10.3389/fpsyg.2013.00907 10.1073/pnas.1613844114 10.1016/j.neunet.2009.12.007 10.1016/j.cub.2009.06.021 10.1016/j.neuron.2020.09.020 10.1016/j.cub.2022.03.064 10.1038/nn1560 10.1371/journal.pcbi.1009070 10.1109/TAMD.2010.2051031 10.1038/s41467-021-25123-3 10.1126/science.abf4588 10.1523/JNEUROSCI.5331-09.2010 10.7554/eLife.21886 10.1038/nn.2682 10.1037/bne0000049 10.1152/jn.1992.67.1.145 10.1073/pnas.1911778117 10.1037/h0048640 10.1523/JNEUROSCI.4171-13.2014 10.3390/e25020327 10.1523/JNEUROSCI.2816-17.2018 10.7554/eLife.66358 10.1016/j.cognition.2022.105119 10.1146/annurev.ph.25.030163.002553 10.1111/ejn.13106 10.1016/S0006-8993(97)01004-4 10.1038/s41562-019-0589-3 10.1523/JNEUROSCI.20-01-00451.2000 10.1016/0926-6410(95)90007-1 10.1016/j.neuron.2014.08.060 10.1016/j.conb.2023.102758 10.1152/jn.1993.69.6.1918 10.1016/j.neuron.2020.01.012 10.1007/BF00229353 10.1016/0306-4522(94)00578-S 10.1038/s41467-019-13135-z 10.1038/s41583-018-0078-0 10.1038/s41593-019-0440-1 10.1111/j.1460-9568.1995.tb01108.x 10.1016/j.neuron.2014.12.050 10.1016/j.neuron.2006.06.021 10.1038/s41562-023-01661-2 10.1016/j.conb.2011.11.013 10.1016/j.neuron.2022.08.022 10.1038/s41593-021-00950-1 10.1073/pnas.0308528101 10.1007/BF00234113 10.1038/s41467-017-02038-6 10.1016/j.conb.2019.08.004 10.1016/j.visres.2008.09.007 10.1152/jn.1983.49.5.1254 10.1016/j.conb.2018.11.003 10.1111/j.1467-9280.2009.02402.x 10.1126/science.959831 10.1101/lm.82504 10.1038/nature19325 10.1037/rev0000175 10.1016/S0893-6080(02)00048-5
ContentType	Journal Article
Copyright	2023 The Authors Copyright © 2023 The Authors. Published by Elsevier Ltd.. All rights reserved.
Copyright_xml	– notice: 2023 The Authors – notice: Copyright © 2023 The Authors. Published by Elsevier Ltd.. All rights reserved.
DBID	AAYXX CITATION CGR CUY CVF ECM EIF NPM 7X8
DOI	10.1016/j.tins.2023.10.002
DatabaseName	CrossRef Medline MEDLINE MEDLINE (Ovid) MEDLINE MEDLINE PubMed MEDLINE - Academic
DatabaseTitle	CrossRef MEDLINE Medline Complete MEDLINE with Full Text PubMed MEDLINE (Ovid) MEDLINE - Academic
DatabaseTitleList	MEDLINE MEDLINE - Academic
Database_xml	– sequence: 1 dbid: NPM name: PubMed url: http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed sourceTypes: Index Database – sequence: 2 dbid: 7X8 name: MEDLINE - Academic url: https://search.proquest.com/medline sourceTypes: Aggregation Database
DeliveryMethod	fulltext_linktorsrc
Discipline	Medicine Anatomy & Physiology
EISSN	1878-108X
EndPage	1066
ExternalDocumentID	37925342 10_1016_j_tins_2023_10_002 S0166223623002400
Genre	Journal Article Review
GroupedDBID	--- --K --M -DZ -~X .1- .55 .FO .GJ .~1 0R~ 123 1B1 1CY 1P~ 1RT 1~. 1~5 29Q 3O- 4.4 457 4G. 53G 5VS 7-5 71M 8P~ 9JM 9M8 AABNK AAEDT AAEDW AAIKJ AAKOC AALRI AAMRU AAOAW AAQFI AAQXK AATTM AAXKI AAXLA AAXUO AAYWO ABCQJ ABFNM ABIVO ABJNI ABLJU ABMAC ABTEW ABUFD ABWVN ABXDB ACDAQ ACGFS ACIUM ACIWK ACLOT ACPRK ACRLP ACRPL ACVFH ADBBV ADCNI ADEZE ADIYS ADMUD ADNMO ADXHL AEBSH AEIPS AEKER AENEX AEUPX AEVXI AFPUW AFRAH AFRHN AFTJW AFXIZ AGHFR AGQPQ AGUBO AGWIK AGYEJ AHHHB AHMBA AIEXJ AIGII AIIUN AIKHN AITUG AJUYK AKBMS AKRWK AKYEP ALMA_UNASSIGNED_HOLDINGS AMRAJ ANKPU APXCP ASPBG AVWKF AXJTR AZFZN BKOJK BKOMP BLXMC CS3 DU5 EBS EFJIC EFKBS EFLBG EJD EO8 EO9 EP2 EP3 F5P FDB FEDTE FGOYB FIRID FNPLU FYGXN G-2 G-Q GBLVA HMQ HVGLF HZ~ IHE J1W KOM M2V M41 MO0 MOBAO MVM N9A O-L O9- OAUVE OHT OP~ OZT P-8 P-9 P2P PC. PQQKQ Q38 R2- ROL RPZ RXW SCC SDF SDG SDP SES SEW SNS SPCBC SSN SSZ T5K TAE TN5 UQL WH7 WUQ X7M XPP YYP Z5R ZCA ZGI ZKB ~G- ~HD 9DU AAYXX CITATION AFCTW AGCQF AGRNS BNPGV CGR CUY CVF ECM EIF NPM RIG SSH 7X8
ID	FETCH-LOGICAL-c451t-4e89a42d07e75a7a1e25718b0c31f2aadde89c7379d55b84807118cc8f2d660f3
ISICitedReferencesCount	26
ISICitedReferencesURI	http://www.webofscience.com/api/gateway?GWVersion=2&SrcApp=Summon&SrcAuth=ProQuest&DestLinkType=CitingArticles&DestApp=WOS_CPL&KeyUT=001123389800001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
ISSN	0166-2236 1878-108X
IngestDate	Sat Sep 27 18:16:20 EDT 2025 Mon Jul 21 05:45:28 EDT 2025 Tue Nov 18 19:58:27 EST 2025 Sat Nov 29 07:25:44 EST 2025 Tue Oct 14 19:30:49 EDT 2025
IsDoiOpenAccess	true
IsOpenAccess	true
IsPeerReviewed	true
IsScholarly	true
Issue	12
Keywords	surprise orienting novelty neural circuits information reinforcement learning neuromodulatory systems
Language	English
License	This is an open access article under the CC BY-NC-ND license. Copyright © 2023 The Authors. Published by Elsevier Ltd.. All rights reserved.
LinkModel	OpenURL
MergedId	FETCHMERGED-LOGICAL-c451t-4e89a42d07e75a7a1e25718b0c31f2aadde89c7379d55b84807118cc8f2d660f3
Notes	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 ObjectType-Review-3 content type line 23
ORCID	0000-0003-4924-7381
OpenAccessLink	https://www.clinicalkey.com/#!/content/1-s2.0-S0166223623002400
PMID	37925342
PQID	2886329167
PQPubID	23479
PageCount	13
ParticipantIDs	proquest_miscellaneous_2886329167 pubmed_primary_37925342 crossref_primary_10_1016_j_tins_2023_10_002 crossref_citationtrail_10_1016_j_tins_2023_10_002 elsevier_clinicalkey_doi_10_1016_j_tins_2023_10_002
PublicationCentury	2000
PublicationDate	December 2023 2023-12-00 20231201
PublicationDateYYYYMMDD	2023-12-01
PublicationDate_xml	– month: 12 year: 2023 text: December 2023
PublicationDecade	2020
PublicationPlace	England
PublicationPlace_xml	– name: England
PublicationTitle	Trends in neurosciences (Regular ed.)
PublicationTitleAlternate	Trends Neurosci
PublicationYear	2023
Publisher	Elsevier Ltd
Publisher_xml	– name: Elsevier Ltd
References	Bromberg-Martin, Hikosaka (bb0315) 2009; 63 Gottlieb, Oudeyer (bb0075) 2018; 19 Li (bb0100) 1993; 69 Deschênes (bb0050) 2012; 22 Schulz, Gershman (bb0425) 2019; 55 Foley (bb0365) 2017; 114 Bellemare (bb0480) 2016; 29 Berlyne (bb0020) 1954; 45 Nieuwenhuis (bb0170) 2011; 48 Mendonca (bb0495) 2021; 34 Kobayashi (bb0305) 2019; 3 Dubey, Griffiths (bb0580) 2019; 127 Singh (bb0475) 2010; 2 Honey (bb0150) 1998; 18 Baranes (bb0320) 2015; 117 van Lieshout (bb0385) 2018; 38 Vankov (bb0220) 1995; 7 Rutishauser (bb0125) 2006; 49 Morrens (bb0245) 2020; 106 Daw (bb0535) 2005; 8 Kato (bb0105) 2012; 76 Kim (bb0525) 2019; 97 Wittmann (bb0335) 2007; 38 Steinmetz (bb0600) 2021; 372 Schmidhuber (bb0460) 2010; 2 Sokolov (bb0045) 1963; 25 O’Doherty (bb0530) 2021; 123 Pavlov, Anrep (bb0010) 2015 Nelson (bb0550) 2005; 112 McElvain (bb0190) 2018; 368 Modirshanechi (bb0240) 2019; 196 Barto (bb0625) 2013; 4 Xiang, Brown (bb0120) 1998; 37 Alet (bb0595) 2020 Sara (bb0230) 1995; 2 Bromberg-Martin (bb0455) 2022 Baldi, Itti (bb0505) 2010; 23 Wilson (bb0395) 2014; 143 Valdés-Baizabal (bb0285) 2020; 18 Schomaker, Meeter (bb0235) 2018; 1694 Sutton, Barto (bb0470) 2018 Crane (bb0080) 2020; 95 Ten (bb0295) 2021; 12 Loewenstein (bb0040) 1994; 116 Pathak (bb0565) 2019; 97 Schultz (bb0330) 1997; 275 Itti, Baldi (bb0500) 2009; 49 Sekar (bb0490) 2020; 119 Levy (bb0155) 2004; 11 Horvath (bb0545) 2021; 4 Murayama (bb0465) 2022; 129 Xu (bb0390) 2021; 17 Guitart-Masip (bb0340) 2010; 30 Fantz (bb0060) 1964; 146 Carter (bb0215) 2010; 13 Dubois (bb0400) 2021; 10 Kamiński (bb0265) 2018; 28 Schiemann (bb0270) 2012; 15 Jaegle (bb0630) 2019; 58 Oudeyer (bb0435) 2018 Esquivelzeta Rabell (bb0055) 2017; 27 Da Silva (bb0275) 2018; 554 Squires (bb0620) 1976; 193 Modirshanechi (bb0415) 2022 Gershman, Niv (bb0420) 2015; 7 Zola (bb0165) 2000; 20 Wu (bb0430) 2018; 2 Binz, Schulz (bb0585) 2022; 35 Kakade, Dayan (bb0515) 2002; 15 Blanchard (bb0410) 2015; 85 Bunzeck (bb0065) 2009; 19 Montgomery (bb0440) 1954; 47 Scourse, Hinde (bb0200) 1973; 47 Lak (bb0260) 2016; 5 Parras (bb0095) 2017; 8 White (bb0360) 2019; 10 Wittmann (bb0345) 2008; 58 Aubret (bb0445) 2023; 25 Bunzeck, Düzel (bb0255) 2006; 51 Hunt (bb0035) 1963; 1963 Lau (bb0405) 2020; 4 Akiti (bb0280) 2022; 110 Bromberg-Martin, Hikosaka (bb0325) 2011; 14 Hebb (bb0030) 1955; 62 Ogasawara (bb0290) 2022; 25 Horan (bb0375) 2019; 22 Cogliati Dezza (bb0520) 2022; 11 Inglis, Fibiger (bb0195) 1995; 66 Hikosaka, Wurtz (bb0180) 1983; 49 Antony (bb0615) 2021; 109 Olarte-Sánchez (bb0145) 2015; 42 Heyes, Huber (bb0025) 2000 Rebec (bb0250) 1997; 776 Li (bb0380) 2022; 13 Ostrovski (bb0485) 2017 Brunner (bb0605) 2020; 108 Foley (bb0370) 2014; 34 Klyubin (bb0560) 2005; 1 Modirshanechi (bb0610) 2022; 110 James (bb0005) 1899 Modirshanechi (bb0085) 2023; 82 Prusky (bb0160) 2004; 101 Takeuchi (bb0225) 2016; 537 Gruber (bb0310) 2014; 84 Dashiell (bb0015) 1925; 5 Piray, Daw (bb0540) 2021; 12 Zheng (bb0590) 2020; 119 Albasser (bb0140) 2015; 129 Bouret, Richmond (bb0205) 2008; 101 Mohamed, Jimenez Rezende (bb0570) 2015; 28 Schomaker, Meeter (bb0090) 2015; 55 Poli (bb0510) 2022; 225 Fahy (bb0115) 1993; 96 Ahmadlou (bb0070) 2021; 372 Menegas (bb0355) 2017; 6 Wilson, Rolls (bb0130) 1993; 93 Strehl, Littman (bb0575) 2008; 74 Ljungberg (bb0210) 1992; 67 Brändle (bb0555) 2023; 7 Zhang (bb0110) 2022; 32 Hikosaka, Wurtz (bb0185) 1983; 49 Cen (bb0300) 2021; 28 Kang (bb0350) 2009; 20 Allen (bb0175) 2021; 71 Filimon (bb0450) 2020; 117 Thompson, Welker (bb0135) 1963; 56 Montgomery (10.1016/j.tins.2023.10.002_bb0440) 1954; 47 O’Doherty (10.1016/j.tins.2023.10.002_bb0530) 2021; 123 Li (10.1016/j.tins.2023.10.002_bb0100) 1993; 69 Da Silva (10.1016/j.tins.2023.10.002_bb0275) 2018; 554 Wittmann (10.1016/j.tins.2023.10.002_bb0345) 2008; 58 Wilson (10.1016/j.tins.2023.10.002_bb0395) 2014; 143 Ostrovski (10.1016/j.tins.2023.10.002_bb0485) 2017 Pavlov (10.1016/j.tins.2023.10.002_bb0010) 2015 Modirshanechi (10.1016/j.tins.2023.10.002_bb0415) 2022 Fahy (10.1016/j.tins.2023.10.002_bb0115) 1993; 96 Barto (10.1016/j.tins.2023.10.002_bb0625) 2013; 4 Bouret (10.1016/j.tins.2023.10.002_bb0205) 2008; 101 Kang (10.1016/j.tins.2023.10.002_bb0350) 2009; 20 Aubret (10.1016/j.tins.2023.10.002_bb0445) 2023; 25 Modirshanechi (10.1016/j.tins.2023.10.002_bb0240) 2019; 196 Bunzeck (10.1016/j.tins.2023.10.002_bb0065) 2009; 19 Vankov (10.1016/j.tins.2023.10.002_bb0220) 1995; 7 Modirshanechi (10.1016/j.tins.2023.10.002_bb0085) 2023; 82 Nelson (10.1016/j.tins.2023.10.002_bb0550) 2005; 112 Bromberg-Martin (10.1016/j.tins.2023.10.002_bb0455) 2022 Singh (10.1016/j.tins.2023.10.002_bb0475) 2010; 2 Zhang (10.1016/j.tins.2023.10.002_bb0110) 2022; 32 Hikosaka (10.1016/j.tins.2023.10.002_bb0185) 1983; 49 Prusky (10.1016/j.tins.2023.10.002_bb0160) 2004; 101 Berlyne (10.1016/j.tins.2023.10.002_bb0020) 1954; 45 Oudeyer (10.1016/j.tins.2023.10.002_bb0435) 2018 Piray (10.1016/j.tins.2023.10.002_bb0540) 2021; 12 Filimon (10.1016/j.tins.2023.10.002_bb0450) 2020; 117 Mendonca (10.1016/j.tins.2023.10.002_bb0495) 2021; 34 Ljungberg (10.1016/j.tins.2023.10.002_bb0210) 1992; 67 Lak (10.1016/j.tins.2023.10.002_bb0260) 2016; 5 Esquivelzeta Rabell (10.1016/j.tins.2023.10.002_bb0055) 2017; 27 Strehl (10.1016/j.tins.2023.10.002_bb0575) 2008; 74 Schomaker (10.1016/j.tins.2023.10.002_bb0235) 2018; 1694 Ten (10.1016/j.tins.2023.10.002_bb0295) 2021; 12 Bromberg-Martin (10.1016/j.tins.2023.10.002_bb0325) 2011; 14 Crane (10.1016/j.tins.2023.10.002_bb0080) 2020; 95 Li (10.1016/j.tins.2023.10.002_bb0380) 2022; 13 Rutishauser (10.1016/j.tins.2023.10.002_bb0125) 2006; 49 McElvain (10.1016/j.tins.2023.10.002_bb0190) 2018; 368 Morrens (10.1016/j.tins.2023.10.002_bb0245) 2020; 106 Carter (10.1016/j.tins.2023.10.002_bb0215) 2010; 13 Baranes (10.1016/j.tins.2023.10.002_bb0320) 2015; 117 Antony (10.1016/j.tins.2023.10.002_bb0615) 2021; 109 Loewenstein (10.1016/j.tins.2023.10.002_bb0040) 1994; 116 Dashiell (10.1016/j.tins.2023.10.002_bb0015) 1925; 5 Wilson (10.1016/j.tins.2023.10.002_bb0130) 1993; 93 Kato (10.1016/j.tins.2023.10.002_bb0105) 2012; 76 Foley (10.1016/j.tins.2023.10.002_bb0365) 2017; 114 Pathak (10.1016/j.tins.2023.10.002_bb0565) 2019; 97 Kim (10.1016/j.tins.2023.10.002_bb0525) 2019; 97 Gruber (10.1016/j.tins.2023.10.002_bb0310) 2014; 84 Horan (10.1016/j.tins.2023.10.002_bb0375) 2019; 22 Cogliati Dezza (10.1016/j.tins.2023.10.002_bb0520) 2022; 11 Zheng (10.1016/j.tins.2023.10.002_bb0590) 2020; 119 Ahmadlou (10.1016/j.tins.2023.10.002_bb0070) 2021; 372 Takeuchi (10.1016/j.tins.2023.10.002_bb0225) 2016; 537 Sutton (10.1016/j.tins.2023.10.002_bb0470) 2018 Kakade (10.1016/j.tins.2023.10.002_bb0515) 2002; 15 Xiang (10.1016/j.tins.2023.10.002_bb0120) 1998; 37 Jaegle (10.1016/j.tins.2023.10.002_bb0630) 2019; 58 Hebb (10.1016/j.tins.2023.10.002_bb0030) 1955; 62 Schulz (10.1016/j.tins.2023.10.002_bb0425) 2019; 55 Deschênes (10.1016/j.tins.2023.10.002_bb0050) 2012; 22 Baldi (10.1016/j.tins.2023.10.002_bb0505) 2010; 23 Itti (10.1016/j.tins.2023.10.002_bb0500) 2009; 49 Gershman (10.1016/j.tins.2023.10.002_bb0420) 2015; 7 Wu (10.1016/j.tins.2023.10.002_bb0430) 2018; 2 Heyes (10.1016/j.tins.2023.10.002_bb0025) 2000 Wittmann (10.1016/j.tins.2023.10.002_bb0335) 2007; 38 Inglis (10.1016/j.tins.2023.10.002_bb0195) 1995; 66 Sokolov (10.1016/j.tins.2023.10.002_bb0045) 1963; 25 Dubey (10.1016/j.tins.2023.10.002_bb0580) 2019; 127 Valdés-Baizabal (10.1016/j.tins.2023.10.002_bb0285) 2020; 18 Honey (10.1016/j.tins.2023.10.002_bb0150) 1998; 18 Ogasawara (10.1016/j.tins.2023.10.002_bb0290) 2022; 25 Binz (10.1016/j.tins.2023.10.002_bb0585) 2022; 35 Cen (10.1016/j.tins.2023.10.002_bb0300) 2021; 28 Bromberg-Martin (10.1016/j.tins.2023.10.002_bb0315) 2009; 63 Lau (10.1016/j.tins.2023.10.002_bb0405) 2020; 4 Alet (10.1016/j.tins.2023.10.002_bb0595) 2020 Albasser (10.1016/j.tins.2023.10.002_bb0140) 2015; 129 James (10.1016/j.tins.2023.10.002_bb0005) 1899 Scourse (10.1016/j.tins.2023.10.002_bb0200) 1973; 47 Parras (10.1016/j.tins.2023.10.002_bb0095) 2017; 8 Horvath (10.1016/j.tins.2023.10.002_bb0545) 2021; 4 Sekar (10.1016/j.tins.2023.10.002_bb0490) 2020; 119 Nieuwenhuis (10.1016/j.tins.2023.10.002_bb0170) 2011; 48 Rebec (10.1016/j.tins.2023.10.002_bb0250) 1997; 776 Foley (10.1016/j.tins.2023.10.002_bb0370) 2014; 34 Allen (10.1016/j.tins.2023.10.002_bb0175) 2021; 71 Murayama (10.1016/j.tins.2023.10.002_bb0465) 2022; 129 Bellemare (10.1016/j.tins.2023.10.002_bb0480) 2016; 29 Levy (10.1016/j.tins.2023.10.002_bb0155) 2004; 11 Olarte-Sánchez (10.1016/j.tins.2023.10.002_bb0145) 2015; 42 Klyubin (10.1016/j.tins.2023.10.002_bb0560) 2005; 1 Dubois (10.1016/j.tins.2023.10.002_bb0400) 2021; 10 Sara (10.1016/j.tins.2023.10.002_bb0230) 1995; 2 Steinmetz (10.1016/j.tins.2023.10.002_bb0600) 2021; 372 Brunner (10.1016/j.tins.2023.10.002_bb0605) 2020; 108 Guitart-Masip (10.1016/j.tins.2023.10.002_bb0340) 2010; 30 Poli (10.1016/j.tins.2023.10.002_bb0510) 2022; 225 Schiemann (10.1016/j.tins.2023.10.002_bb0270) 2012; 15 Schmidhuber (10.1016/j.tins.2023.10.002_bb0460) 2010; 2 Bunzeck (10.1016/j.tins.2023.10.002_bb0255) 2006; 51 van Lieshout (10.1016/j.tins.2023.10.002_bb0385) 2018; 38 Hunt (10.1016/j.tins.2023.10.002_bb0035) 1963; 1963 Hikosaka (10.1016/j.tins.2023.10.002_bb0180) 1983; 49 Schomaker (10.1016/j.tins.2023.10.002_bb0090) 2015; 55 Brändle (10.1016/j.tins.2023.10.002_bb0555) 2023; 7 Daw (10.1016/j.tins.2023.10.002_bb0535) 2005; 8 Gottlieb (10.1016/j.tins.2023.10.002_bb0075) 2018; 19 Akiti (10.1016/j.tins.2023.10.002_bb0280) 2022; 110 Modirshanechi (10.1016/j.tins.2023.10.002_bb0610) 2022; 110 Mohamed (10.1016/j.tins.2023.10.002_bb0570) 2015; 28 Menegas (10.1016/j.tins.2023.10.002_bb0355) 2017; 6 Kobayashi (10.1016/j.tins.2023.10.002_bb0305) 2019; 3 White (10.1016/j.tins.2023.10.002_bb0360) 2019; 10 Kamiński (10.1016/j.tins.2023.10.002_bb0265) 2018; 28 Fantz (10.1016/j.tins.2023.10.002_bb0060) 1964; 146 Blanchard (10.1016/j.tins.2023.10.002_bb0410) 2015; 85 Zola (10.1016/j.tins.2023.10.002_bb0165) 2000; 20 Squires (10.1016/j.tins.2023.10.002_bb0620) 1976; 193 Thompson (10.1016/j.tins.2023.10.002_bb0135) 1963; 56 Schultz (10.1016/j.tins.2023.10.002_bb0330) 1997; 275 Xu (10.1016/j.tins.2023.10.002_bb0390) 2021; 17
References_xml	– volume: 35 start-page: 31755 year: 2022 end-page: 31768 ident: bb0585 article-title: Modeling human exploration through resource-rational reinforcement learning publication-title: Advances in Neural Information Processing Systems – volume: 56 start-page: 996 year: 1963 end-page: 1002 ident: bb0135 article-title: Role of auditory cortex in reflex head orientation by cats to auditory stimuli publication-title: J. Comp. Physiol. Psychol. – volume: 372 start-page: eabf4588 year: 2021 ident: bb0600 article-title: Neuropixels 2.0: a miniaturized high-density probe for stable, long-term brain recordings publication-title: Science – volume: 96 start-page: 457 year: 1993 end-page: 472 ident: bb0115 article-title: Neuronal activity related to visual recognition memory: long-term memory and the encoding of recency and familiarity information in the primate anterior and medial inferior temporal and rhinal cortex publication-title: Exp. Brain Res. – volume: 117 start-page: 15200 year: 2020 end-page: 15208 ident: bb0450 article-title: The ventral striatum dissociates information expectation, reward anticipation, and reward receipt publication-title: Proc. Natl. Acad. Sci. – volume: 4 start-page: 907 year: 2013 ident: bb0625 article-title: Novelty or Surprise? publication-title: Front. Psychol. – volume: 5 start-page: 205 year: 1925 ident: bb0015 article-title: A quantitative demonstration of animal drive publication-title: J. Comp. Psychol. – volume: 48 start-page: 162 year: 2011 end-page: 175 ident: bb0170 article-title: The anatomical and functional relationship between the P3 and autonomic components of the orienting response publication-title: Psychophysiology – volume: 119 start-page: 8583 year: 2020 end-page: 8592 ident: bb0490 article-title: Planning to Explore via Self-Supervised World Models publication-title: Proceedings of Machine Learning Research – volume: 49 start-page: 1295 year: 2009 end-page: 1306 ident: bb0500 article-title: Bayesian surprise attracts human attention publication-title: Vis. Res. – year: 2000 ident: bb0025 article-title: The Evolution of Cognition – volume: 82 year: 2023 ident: bb0085 article-title: Surprise and novelty in the brain publication-title: Curr. Opin. Neurobiol. – volume: 22 start-page: 1327 year: 2019 end-page: 1335 ident: bb0375 article-title: Parietal neurons encode information sampling based on decision uncertainty publication-title: Nat. Neurosci. – volume: 15 start-page: 1272 year: 2012 end-page: 1280 ident: bb0270 article-title: K-ATP channels in dopamine substantia nigra neurons control bursting and novelty-induced exploration publication-title: Nat. Neurosci. – volume: 47 start-page: 1 year: 1973 end-page: 13 ident: bb0200 article-title: Habituation to auditory stimuli in mice publication-title: Behaviour – volume: 45 start-page: 180 year: 1954 end-page: 191 ident: bb0020 article-title: A theory of human curiosity publication-title: Br. J. Psychol. Gen. Sect. – volume: 23 start-page: 649 year: 2010 end-page: 666 ident: bb0505 article-title: Of bits and wows: a Bayesian theory of surprise with applications to attention publication-title: Neural Netw. – volume: 19 start-page: 1294 year: 2009 end-page: 1300 ident: bb0065 article-title: Reward motivation accelerates the onset of neural novelty signals in humans to 85 milliseconds publication-title: Curr. Biol. CB – year: 2017 ident: bb0485 article-title: Count-based exploration with neural density models publication-title: Proceedings of the 34th International Conference on Machine Learning – year: 2020 ident: bb0595 article-title: Meta-learning curiosity algorithms publication-title: International Conference on Learning Representations – volume: 20 start-page: 451 year: 2000 end-page: 463 ident: bb0165 article-title: Impaired recognition memory in monkeys after damage limited to the hippocampal region publication-title: J. Neurosci. – volume: 106 start-page: 142 year: 2020 end-page: 153.e7 ident: bb0245 article-title: Cue-evoked dopamine promotes conditioned responding during learning publication-title: Neuron – volume: 12 start-page: 5972 year: 2021 ident: bb0295 article-title: Humans monitor learning progress in curiosity-driven exploration publication-title: Nat. Commun. – volume: 13 start-page: 1526 year: 2010 end-page: 1533 ident: bb0215 article-title: Tuning arousal with optogenetic modulation of locus coeruleus neurons publication-title: Nat. Neurosci. – volume: 2 start-page: 70 year: 2010 end-page: 82 ident: bb0475 article-title: Intrinsically motivated reinforcement learning: an evolutionary perspective publication-title: IEEE Trans. Auton. Ment. Dev. – volume: 112 start-page: 979 year: 2005 end-page: 999 ident: bb0550 article-title: Finding useful questions: on Bayesian diagnosticity, probability, impact, and information gain publication-title: Psychol. Rev. – volume: 114 start-page: E3315 year: 2017 end-page: E3323 ident: bb0365 article-title: Parietal neurons encode expected gains in instrumental information publication-title: Proc. Natl. Acad. Sci. – volume: 11 start-page: 794 year: 2004 end-page: 796 ident: bb0155 article-title: Impaired odor recognition memory in patients with hippocampal lesions publication-title: Learn. Mem. – volume: 368 start-page: 152 year: 2018 end-page: 170 ident: bb0190 article-title: Circuits in the rodent brainstem that control whisking in concert with other orofacial motor actions publication-title: Neuroscience – year: 2022 ident: bb0415 article-title: The curse of optimism: a persistent distraction by novelty publication-title: bioRxiv – volume: 1694 start-page: 140 year: 2018 end-page: 148 ident: bb0235 article-title: Predicting the unknown: novelty processing depends on expectations publication-title: Brain Res. – volume: 30 start-page: 1721 year: 2010 end-page: 1726 ident: bb0340 article-title: Contextual novelty changes reward representations in the striatum publication-title: J. Neurosci. – volume: 109 start-page: 377 year: 2021 end-page: 390.e7 ident: bb0615 article-title: Behavioral, physiological, and neural signatures of surprise during naturalistic sports viewing publication-title: Neuron – volume: 22 start-page: 243 year: 2012 end-page: 250 ident: bb0050 article-title: Sniffing and whisking in rodents publication-title: Curr. Opin. Neurobiol. – volume: 97 start-page: 3379 year: 2019 end-page: 3388 ident: bb0525 article-title: Curiosity-bottleneck: exploration by distilling task-specific novelty publication-title: Proceedings of the 36th International Conference on Machine Learning – volume: 4 start-page: 442 year: 2021 end-page: 462 ident: bb0545 article-title: Human belief state-based exploration and exploitation in an information-selective symmetric reversal bandit task publication-title: Comput. Brain Behav. – volume: 8 start-page: 1704 year: 2005 end-page: 1711 ident: bb0535 article-title: Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control publication-title: Nat. Neurosci. – volume: 25 start-page: 545 year: 1963 end-page: 580 ident: bb0045 article-title: Higher nervous functions: the orienting reflex publication-title: Annu. Rev. Physiol. – volume: 10 start-page: 5168 year: 2019 ident: bb0360 article-title: A neural network for information seeking publication-title: Nat. Commun. – volume: 146 start-page: 668 year: 1964 end-page: 670 ident: bb0060 article-title: Visual experience in infants: decreased attention to familiar patterns relative to novel ones publication-title: Science – volume: 38 start-page: 2579 year: 2018 end-page: 2588 ident: bb0385 article-title: Induction and relief of curiosity elicit parietal and frontal activity publication-title: J. Neurosci. – volume: 101 start-page: 898 year: 2008 end-page: 911 ident: bb0205 article-title: Relation of locus coeruleus neurons in monkeys to Pavlovian and operant behaviors publication-title: J. Neurophysiol. – volume: 372 year: 2021 ident: bb0070 article-title: A cell type-specific cortico-subcortical brain circuit for investigatory and novelty-seeking behavior publication-title: Science – volume: 14 start-page: 1209 year: 2011 end-page: 1216 ident: bb0325 article-title: Lateral habenula neurons signal errors in the prediction of reward information publication-title: Nat. Neurosci. – volume: 108 start-page: 861 year: 2020 end-page: 875.e7 ident: bb0605 article-title: A platform for brain-wide volumetric functional ultrasound imaging and analysis of circuit dynamics in awake mice publication-title: Neuron – volume: 117 start-page: 81 year: 2015 end-page: 90 ident: bb0320 article-title: Eye movements reveal epistemic curiosity in human observers publication-title: Vis. Res. – volume: 20 start-page: 963 year: 2009 end-page: 973 ident: bb0350 article-title: The wick in the candle of learning: epistemic curiosity activates reward circuitry and enhances memory publication-title: Psychol. Sci. – volume: 76 start-page: 962 year: 2012 end-page: 975 ident: bb0105 article-title: Dynamic sensory representations in the olfactory bulb: modulation by wakefulness and experience publication-title: Neuron – year: 1899 ident: bb0005 article-title: Talks to Teachers on Psychology – and to Students on Some of Life’s Ideals – volume: 38 start-page: 194 year: 2007 end-page: 202 ident: bb0335 article-title: Anticipation of novelty recruits reward system and hippocampus while promoting recollection publication-title: NeuroImage – volume: 95 start-page: 218 year: 2020 end-page: 231 ident: bb0080 article-title: An ecological framework of neophobia: from cells to organisms to populations publication-title: Biol. Rev. – volume: 129 start-page: 175 year: 2022 end-page: 198 ident: bb0465 article-title: A reward-learning framework of knowledge acquisition: an integrated account of curiosity, interest, and intrinsic–extrinsic rewards publication-title: Psychol. Rev. – volume: 49 start-page: 1285 year: 1983 end-page: 1301 ident: bb0185 article-title: Visual and oculomotor functions of monkey substantia nigra pars reticulata. IV. Relation of substantia nigra to superior colliculus publication-title: J. Neurophysiol. – volume: 1963 start-page: 35 year: 1963 end-page: 94 ident: bb0035 article-title: Motivation inherent in information processing and action publication-title: Motiv. Soc. Interaction – volume: 66 start-page: 81 year: 1995 end-page: 86 ident: bb0195 article-title: Increases in hippocampal and frontal cortical acetylcholine release associated with presentation of sensory stimuli publication-title: Neuroscience – volume: 129 start-page: 227 year: 2015 end-page: 243 ident: bb0140 article-title: Perirhinal cortex lesions in rats: novelty detection and sensitivity to interference publication-title: Behav. Neurosci. – volume: 116 start-page: 75 year: 1994 end-page: 98 ident: bb0040 article-title: The psychology of curiosity: a review and reinterpretation publication-title: Psychol. Bull. – year: 2018 ident: bb0470 article-title: Reinforcement Learning: An Introduction – volume: 85 start-page: 602 year: 2015 end-page: 614 ident: bb0410 article-title: Orbitofrontal cortex uses distinct codes for different choice attributes in decisions motivated by curiosity publication-title: Neuron – volume: 18 year: 2020 ident: bb0285 article-title: Dopamine modulates subcortical responses to surprising sounds publication-title: PLoS Biol. – volume: 275 start-page: 1593 year: 1997 end-page: 1599 ident: bb0330 article-title: A neural substrate of prediction and reward publication-title: Science – volume: 84 start-page: 486 year: 2014 end-page: 496 ident: bb0310 article-title: States of curiosity modulate hippocampus-dependent learning via the dopaminergic circuit publication-title: Neuron – year: 2018 ident: bb0435 article-title: Computational theories of curiosity-driven learning publication-title: arXiv – volume: 55 start-page: 268 year: 2015 end-page: 279 ident: bb0090 article-title: Short- and long-lasting consequences of novelty, deviance and surprise on brain and cognition publication-title: Neurosci. Biobehav. Rev. – volume: 4 start-page: 531 year: 2020 end-page: 543 ident: bb0405 article-title: Shared striatal activity in decisions to satisfy curiosity and hunger at the risk of electric shocks publication-title: Nat. Hum. Behav. – volume: 49 start-page: 805 year: 2006 end-page: 813 ident: bb0125 article-title: Single-trial learning of novel stimuli by individual neurons of the human hippocampus–amygdala complex publication-title: Neuron – volume: 110 start-page: 3789 year: 2022 end-page: 3804.e9 ident: bb0280 article-title: Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction publication-title: Neuron – volume: 123 start-page: 14 year: 2021 end-page: 23 ident: bb0530 article-title: Why and how the brain weights contributions from a mixture of experts publication-title: Neurosci. Biobehav. Rev. – volume: 10 year: 2021 ident: bb0400 article-title: Human complex exploration strategies are enriched by noradrenaline-modulated heuristics publication-title: eLife – volume: 74 start-page: 1309 year: 2008 end-page: 1331 ident: bb0575 article-title: An analysis of model-based interval estimation for Markov decision processes publication-title: J. Comput. Syst. Sci. – volume: 69 start-page: 1918 year: 1993 end-page: 1929 ident: bb0100 article-title: The representation of stimulus familiarity in anterior inferior temporal cortex publication-title: J. Neurophysiol. – volume: 119 start-page: 11436 year: 2020 end-page: 11446 ident: bb0590 article-title: What can learned intrinsic rewards capture? publication-title: Proceedings of the 37th International Conference on Machine Learning – volume: 19 start-page: 758 year: 2018 end-page: 770 ident: bb0075 article-title: Towards a neuroscience of active sampling and curiosity publication-title: Nat. Rev. Neurosci. – volume: 2 start-page: 230 year: 2010 end-page: 247 ident: bb0460 article-title: Formal theory of creativity, fun, and intrinsic motivation (1990–2010) publication-title: IEEE Trans. Auton. Ment. Dev. – volume: 58 start-page: 167 year: 2019 end-page: 174 ident: bb0630 article-title: Visual novelty, curiosity, and intrinsic reward in machine learning and the brain publication-title: Curr. Opin. Neurobiol. – volume: 5 year: 2016 ident: bb0260 article-title: Dopamine neurons learn relative chosen value from probabilistic rewards publication-title: eLife – volume: 25 start-page: 327 year: 2023 ident: bb0445 article-title: An information-theoretic perspective on intrinsic motivation in reinforcement learning: a survey publication-title: Entropy – volume: 2 start-page: 181 year: 1995 end-page: 187 ident: bb0230 article-title: Novelty seeking behavior in the rat is dependent upon the integrity of the noradrenergic system publication-title: Cogn. Brain Res. – volume: 47 start-page: 60 year: 1954 end-page: 64 ident: bb0440 article-title: The role of the exploratory drive in learning publication-title: J. Comp. Physiol. Psychol. – volume: 67 start-page: 145 year: 1992 end-page: 163 ident: bb0210 article-title: Responses of monkey dopamine neurons during learning of behavioral reactions publication-title: J. Neurophysiol. – volume: 93 start-page: 367 year: 1993 end-page: 382 ident: bb0130 article-title: The effects of stimulus novelty and familiarity on neuronal activity in the amygdala of monkeys performing recognition memory tasks publication-title: Exp. Brain Res. – volume: 12 start-page: 4942 year: 2021 ident: bb0540 article-title: Linear reinforcement learning in planning, grid fields, and cognitive control publication-title: Nat. Commun. – volume: 225 year: 2022 ident: bb0510 article-title: Contributions of expected learning progress and perceptual novelty to curiosity-driven exploration publication-title: Cognition – volume: 28 start-page: 2125 year: 2015 end-page: 2133 ident: bb0570 article-title: Variational information maximisation for intrinsically motivated reinforcement learning publication-title: Adv. Neural Inf. Proces. Syst. – volume: 34 start-page: 7947 year: 2014 end-page: 7957 ident: bb0370 article-title: Novelty enhances visual salience independently of reward in the parietal lobe publication-title: J. Neurosci. – volume: 143 start-page: 2074 year: 2014 end-page: 2081 ident: bb0395 article-title: Humans use directed and random exploration to solve the explore–exploit dilemma publication-title: J. Exp. Psychol. Gen. – volume: 8 start-page: 2148 year: 2017 ident: bb0095 article-title: Neurons along the auditory pathway exhibit a hierarchical organization of prediction error publication-title: Nat. Commun. – volume: 776 start-page: 61 year: 1997 end-page: 67 ident: bb0250 article-title: Regional and temporal differences in real-time dopamine efflux in the nucleus accumbens during free-choice novelty publication-title: Brain Res. – volume: 193 start-page: 1142 year: 1976 end-page: 1146 ident: bb0620 article-title: The effect of stimulus sequence on the waveform of the cortical event-related potential publication-title: Science – volume: 110 year: 2022 ident: bb0610 article-title: A taxonomy of surprise definitions publication-title: J. Math. Psychol. – volume: 29 start-page: 1471 year: 2016 end-page: 1479 ident: bb0480 article-title: Unifying count-based exploration and intrinsic motivation publication-title: Adv. Neural Inf. Proces. Syst. – volume: 2 start-page: 915 year: 2018 end-page: 924 ident: bb0430 article-title: Generalization guides human exploration in vast decision spaces publication-title: Nat. Hum. Behav. – volume: 32 start-page: 2160 year: 2022 end-page: 2173.e6 ident: bb0110 article-title: Surprise and recency in novelty detection in the primate brain publication-title: Curr. Biol. – volume: 25 start-page: 50 year: 2022 end-page: 60 ident: bb0290 article-title: A primate temporal cortex–zona incerta pathway for novelty seeking publication-title: Nat. Neurosci. – volume: 28 start-page: 254 year: 2021 end-page: 259 ident: bb0300 article-title: Anticipation of novel environments enhances memory for incidental information publication-title: Learn. Mem. – volume: 196 start-page: 302 year: 2019 end-page: 317 ident: bb0240 article-title: Trial-by-trial surprise-decoding model for visual and auditory binary oddball tasks publication-title: NeuroImage – volume: 97 start-page: 5062 year: 2019 end-page: 5071 ident: bb0565 article-title: Self-supervised exploration via disagreement publication-title: Proceedings of the 36th International Conference on Machine Learning – volume: 6 year: 2017 ident: bb0355 article-title: Opposite initialization to novel cues in dopamine signaling in ventral and posterior striatum in mice publication-title: eLife – year: 2015 ident: bb0010 article-title: Conditioned Reflexes: an Investigation of the Physiological Activity of the Cerebral Cortex – volume: 58 start-page: 967 year: 2008 end-page: 973 ident: bb0345 article-title: Striatal activity underlies novelty-based choice in humans publication-title: Neuron – volume: 55 start-page: 7 year: 2019 end-page: 14 ident: bb0425 article-title: The algorithmic architecture of exploration in the human brain publication-title: Curr. Opin. Neurobiol. – volume: 62 start-page: 243 year: 1955 end-page: 254 ident: bb0030 article-title: Drives and the C. N. S. (conceptual nervous system) publication-title: Psychol. Rev. – volume: 63 start-page: 119 year: 2009 end-page: 126 ident: bb0315 article-title: Midbrain dopamine neurons signal preference for advance information about upcoming rewards publication-title: Neuron – volume: 7 start-page: 1180 year: 1995 end-page: 1187 ident: bb0220 article-title: Response to novelty and its rapid habituation in locus coeruleus neurons of the freely exploring rat publication-title: Eur. J. Neurosci. – volume: 28 start-page: 1333 year: 2018 end-page: 1343.e4 ident: bb0265 article-title: Novelty-sensitive dopaminergic neurons in the human substantia nigra predict success of declarative memory formation publication-title: Curr. Biol. – volume: 11 year: 2022 ident: bb0520 article-title: Independent and interacting value systems for reward and information in the human brain publication-title: eLife – volume: 554 start-page: 244 year: 2018 end-page: 248 ident: bb0275 article-title: Dopamine neuron activity before action initiation gates and invigorates future movements publication-title: Nature – volume: 18 start-page: 2226 year: 1998 end-page: 2230 ident: bb0150 article-title: Hippocampal lesions disrupt an associative mismatch process publication-title: J. Neurosci. – volume: 49 start-page: 1254 year: 1983 end-page: 1267 ident: bb0180 article-title: Visual and oculomotor functions of monkey substantia nigra pars reticulata. II. Visual responses related to fixation of gaze publication-title: J. Neurophysiol. – volume: 3 start-page: 587 year: 2019 end-page: 595 ident: bb0305 article-title: Diverse motives for human curiosity publication-title: Nat. Hum. Behav. – year: 2022 ident: bb0455 article-title: A neural mechanism for conserved value computations integrating information and rewards publication-title: bioRxiv – volume: 7 start-page: 391 year: 2015 end-page: 415 ident: bb0420 article-title: Novelty and inductive generalization in human reinforcement learning publication-title: Top. Cogn. Sci. – volume: 34 start-page: 24379 year: 2021 end-page: 24391 ident: bb0495 article-title: Discovering and achieving goals via world models publication-title: Adv. Neural Inf. Proces. Syst. – volume: 37 start-page: 657 year: 1998 end-page: 676 ident: bb0120 article-title: Differential neuronal encoding of novelty, familiarity and recency in regions of the anterior temporal lobe publication-title: Neuropharmacology – volume: 42 start-page: 3117 year: 2015 end-page: 3127 ident: bb0145 article-title: Perirhinal cortex lesions impair tests of object recognition memory but spare novelty detection publication-title: Eur. J. Neurosci. – volume: 1 start-page: 128 year: 2005 end-page: 135 ident: bb0560 article-title: Empowerment: a universal agent-centric measure of control publication-title: 2005 IEEE Congress on Evolutionary Computation – volume: 127 start-page: 455 year: 2019 end-page: 476 ident: bb0580 article-title: Reconciling novelty and complexity through a rational analysis of curiosity publication-title: Psychol. Rev. – volume: 537 start-page: 357 year: 2016 end-page: 362 ident: bb0225 article-title: Locus coeruleus and dopaminergic consolidation of everyday memory publication-title: Nature – volume: 101 start-page: 5064 year: 2004 end-page: 5068 ident: bb0160 article-title: Visual memory task for rats reveals an essential role for hippocampus and perirhinal cortex publication-title: Proc. Natl. Acad. Sci. – volume: 15 start-page: 549 year: 2002 end-page: 559 ident: bb0515 article-title: Dopamine: generalization and bonuses publication-title: Neural Netw. – volume: 13 start-page: 5911 year: 2022 ident: bb0380 article-title: Uncertainty modulates visual maps during noninstrumental information demand publication-title: Nat. Commun. – volume: 7 start-page: 1481 year: 2023 end-page: 1489 ident: bb0555 article-title: Empowerment contributes to exploration behaviour in a creative video game publication-title: Nat. Hum. Behav. – volume: 71 start-page: 119 year: 2021 end-page: 126 ident: bb0175 article-title: Orienting our view of the superior colliculus: specializations and general functions publication-title: Curr. Opin. Neurobiol. – volume: 17 year: 2021 ident: bb0390 article-title: Novelty is not surprise: human exploratory and adaptive behavior in sequential decision-making publication-title: PLoS Comput. Biol. – volume: 27 start-page: 1542 year: 2017 end-page: 1548.e4 ident: bb0055 article-title: Spontaneous rapid odor source localization behavior requires interhemispheric communication publication-title: Curr. Biol. – volume: 51 start-page: 369 year: 2006 end-page: 379 ident: bb0255 article-title: Absolute coding of stimulus novelty in the human substantia nigra/VTA publication-title: Neuron – year: 2000 ident: 10.1016/j.tins.2023.10.002_bb0025 – volume: 143 start-page: 2074 year: 2014 ident: 10.1016/j.tins.2023.10.002_bb0395 article-title: Humans use directed and random exploration to solve the explore–exploit dilemma publication-title: J. Exp. Psychol. Gen. doi: 10.1037/a0038199 – volume: 47 start-page: 60 year: 1954 ident: 10.1016/j.tins.2023.10.002_bb0440 article-title: The role of the exploratory drive in learning publication-title: J. Comp. Physiol. Psychol. doi: 10.1037/h0054833 – volume: 28 start-page: 1333 year: 2018 ident: 10.1016/j.tins.2023.10.002_bb0265 article-title: Novelty-sensitive dopaminergic neurons in the human substantia nigra predict success of declarative memory formation publication-title: Curr. Biol. doi: 10.1016/j.cub.2018.03.024 – volume: 10 year: 2021 ident: 10.1016/j.tins.2023.10.002_bb0400 article-title: Human complex exploration strategies are enriched by noradrenaline-modulated heuristics publication-title: eLife doi: 10.7554/eLife.59907 – volume: 37 start-page: 657 year: 1998 ident: 10.1016/j.tins.2023.10.002_bb0120 article-title: Differential neuronal encoding of novelty, familiarity and recency in regions of the anterior temporal lobe publication-title: Neuropharmacology doi: 10.1016/S0028-3908(98)00030-6 – volume: 15 start-page: 1272 year: 2012 ident: 10.1016/j.tins.2023.10.002_bb0270 article-title: K-ATP channels in dopamine substantia nigra neurons control bursting and novelty-induced exploration publication-title: Nat. Neurosci. doi: 10.1038/nn.3185 – volume: 110 year: 2022 ident: 10.1016/j.tins.2023.10.002_bb0610 article-title: A taxonomy of surprise definitions publication-title: J. Math. Psychol. doi: 10.1016/j.jmp.2022.102712 – volume: 18 year: 2020 ident: 10.1016/j.tins.2023.10.002_bb0285 article-title: Dopamine modulates subcortical responses to surprising sounds publication-title: PLoS Biol. doi: 10.1371/journal.pbio.3000744 – volume: 372 year: 2021 ident: 10.1016/j.tins.2023.10.002_bb0070 article-title: A cell type-specific cortico-subcortical brain circuit for investigatory and novelty-seeking behavior publication-title: Science doi: 10.1126/science.abe9681 – volume: 27 start-page: 1542 year: 2017 ident: 10.1016/j.tins.2023.10.002_bb0055 article-title: Spontaneous rapid odor source localization behavior requires interhemispheric communication publication-title: Curr. Biol. doi: 10.1016/j.cub.2017.04.027 – volume: 28 start-page: 254 year: 2021 ident: 10.1016/j.tins.2023.10.002_bb0300 article-title: Anticipation of novel environments enhances memory for incidental information publication-title: Learn. Mem. doi: 10.1101/lm.053392.121 – year: 1899 ident: 10.1016/j.tins.2023.10.002_bb0005 – volume: 112 start-page: 979 year: 2005 ident: 10.1016/j.tins.2023.10.002_bb0550 article-title: Finding useful questions: on Bayesian diagnosticity, probability, impact, and information gain publication-title: Psychol. Rev. doi: 10.1037/0033-295X.112.4.979 – volume: 109 start-page: 377 year: 2021 ident: 10.1016/j.tins.2023.10.002_bb0615 article-title: Behavioral, physiological, and neural signatures of surprise during naturalistic sports viewing publication-title: Neuron doi: 10.1016/j.neuron.2020.10.029 – volume: 18 start-page: 2226 year: 1998 ident: 10.1016/j.tins.2023.10.002_bb0150 article-title: Hippocampal lesions disrupt an associative mismatch process publication-title: J. Neurosci. doi: 10.1523/JNEUROSCI.18-06-02226.1998 – volume: 119 start-page: 11436 year: 2020 ident: 10.1016/j.tins.2023.10.002_bb0590 article-title: What can learned intrinsic rewards capture? – volume: 95 start-page: 218 year: 2020 ident: 10.1016/j.tins.2023.10.002_bb0080 article-title: An ecological framework of neophobia: from cells to organisms to populations publication-title: Biol. Rev. doi: 10.1111/brv.12560 – volume: 129 start-page: 175 year: 2022 ident: 10.1016/j.tins.2023.10.002_bb0465 article-title: A reward-learning framework of knowledge acquisition: an integrated account of curiosity, interest, and intrinsic–extrinsic rewards publication-title: Psychol. Rev. doi: 10.1037/rev0000349 – volume: 47 start-page: 1 year: 1973 ident: 10.1016/j.tins.2023.10.002_bb0200 article-title: Habituation to auditory stimuli in mice publication-title: Behaviour doi: 10.1163/156853973X00247 – volume: 116 start-page: 75 year: 1994 ident: 10.1016/j.tins.2023.10.002_bb0040 article-title: The psychology of curiosity: a review and reinterpretation publication-title: Psychol. Bull. doi: 10.1037/0033-2909.116.1.75 – volume: 1694 start-page: 140 year: 2018 ident: 10.1016/j.tins.2023.10.002_bb0235 article-title: Predicting the unknown: novelty processing depends on expectations publication-title: Brain Res. doi: 10.1016/j.brainres.2018.05.008 – year: 2022 ident: 10.1016/j.tins.2023.10.002_bb0455 article-title: A neural mechanism for conserved value computations integrating information and rewards publication-title: bioRxiv – volume: 76 start-page: 962 year: 2012 ident: 10.1016/j.tins.2023.10.002_bb0105 article-title: Dynamic sensory representations in the olfactory bulb: modulation by wakefulness and experience publication-title: Neuron doi: 10.1016/j.neuron.2012.09.037 – volume: 123 start-page: 14 year: 2021 ident: 10.1016/j.tins.2023.10.002_bb0530 article-title: Why and how the brain weights contributions from a mixture of experts publication-title: Neurosci. Biobehav. Rev. doi: 10.1016/j.neubiorev.2020.10.022 – volume: 38 start-page: 194 year: 2007 ident: 10.1016/j.tins.2023.10.002_bb0335 article-title: Anticipation of novelty recruits reward system and hippocampus while promoting recollection publication-title: NeuroImage doi: 10.1016/j.neuroimage.2007.06.038 – volume: 97 start-page: 3379 year: 2019 ident: 10.1016/j.tins.2023.10.002_bb0525 article-title: Curiosity-bottleneck: exploration by distilling task-specific novelty – volume: 5 start-page: 205 year: 1925 ident: 10.1016/j.tins.2023.10.002_bb0015 article-title: A quantitative demonstration of animal drive publication-title: J. Comp. Psychol. doi: 10.1037/h0071833 – volume: 49 start-page: 1285 year: 1983 ident: 10.1016/j.tins.2023.10.002_bb0185 article-title: Visual and oculomotor functions of monkey substantia nigra pars reticulata. IV. Relation of substantia nigra to superior colliculus publication-title: J. Neurophysiol. doi: 10.1152/jn.1983.49.5.1285 – volume: 368 start-page: 152 year: 2018 ident: 10.1016/j.tins.2023.10.002_bb0190 article-title: Circuits in the rodent brainstem that control whisking in concert with other orofacial motor actions publication-title: Neuroscience doi: 10.1016/j.neuroscience.2017.08.034 – volume: 13 start-page: 5911 year: 2022 ident: 10.1016/j.tins.2023.10.002_bb0380 article-title: Uncertainty modulates visual maps during noninstrumental information demand publication-title: Nat. Commun. doi: 10.1038/s41467-022-33585-2 – volume: 5 year: 2016 ident: 10.1016/j.tins.2023.10.002_bb0260 article-title: Dopamine neurons learn relative chosen value from probabilistic rewards publication-title: eLife doi: 10.7554/eLife.18044 – volume: 29 start-page: 1471 year: 2016 ident: 10.1016/j.tins.2023.10.002_bb0480 article-title: Unifying count-based exploration and intrinsic motivation publication-title: Adv. Neural Inf. Proces. Syst. – volume: 62 start-page: 243 year: 1955 ident: 10.1016/j.tins.2023.10.002_bb0030 article-title: Drives and the C. N. S. (conceptual nervous system) publication-title: Psychol. Rev. doi: 10.1037/h0041823 – volume: 12 start-page: 5972 year: 2021 ident: 10.1016/j.tins.2023.10.002_bb0295 article-title: Humans monitor learning progress in curiosity-driven exploration publication-title: Nat. Commun. doi: 10.1038/s41467-021-26196-w – year: 2022 ident: 10.1016/j.tins.2023.10.002_bb0415 article-title: The curse of optimism: a persistent distraction by novelty publication-title: bioRxiv – volume: 2 start-page: 230 year: 2010 ident: 10.1016/j.tins.2023.10.002_bb0460 article-title: Formal theory of creativity, fun, and intrinsic motivation (1990–2010) publication-title: IEEE Trans. Auton. Ment. Dev. doi: 10.1109/TAMD.2010.2056368 – volume: 117 start-page: 81 year: 2015 ident: 10.1016/j.tins.2023.10.002_bb0320 article-title: Eye movements reveal epistemic curiosity in human observers publication-title: Vis. Res. doi: 10.1016/j.visres.2015.10.009 – volume: 7 start-page: 391 year: 2015 ident: 10.1016/j.tins.2023.10.002_bb0420 article-title: Novelty and inductive generalization in human reinforcement learning publication-title: Top. Cogn. Sci. doi: 10.1111/tops.12138 – volume: 63 start-page: 119 year: 2009 ident: 10.1016/j.tins.2023.10.002_bb0315 article-title: Midbrain dopamine neurons signal preference for advance information about upcoming rewards publication-title: Neuron doi: 10.1016/j.neuron.2009.06.009 – volume: 196 start-page: 302 year: 2019 ident: 10.1016/j.tins.2023.10.002_bb0240 article-title: Trial-by-trial surprise-decoding model for visual and auditory binary oddball tasks publication-title: NeuroImage doi: 10.1016/j.neuroimage.2019.04.028 – volume: 2 start-page: 915 year: 2018 ident: 10.1016/j.tins.2023.10.002_bb0430 article-title: Generalization guides human exploration in vast decision spaces publication-title: Nat. Hum. Behav. doi: 10.1038/s41562-018-0467-4 – volume: 58 start-page: 967 year: 2008 ident: 10.1016/j.tins.2023.10.002_bb0345 article-title: Striatal activity underlies novelty-based choice in humans publication-title: Neuron doi: 10.1016/j.neuron.2008.04.027 – volume: 1963 start-page: 35 year: 1963 ident: 10.1016/j.tins.2023.10.002_bb0035 article-title: Motivation inherent in information processing and action – volume: 49 start-page: 805 year: 2006 ident: 10.1016/j.tins.2023.10.002_bb0125 article-title: Single-trial learning of novel stimuli by individual neurons of the human hippocampus–amygdala complex publication-title: Neuron doi: 10.1016/j.neuron.2006.02.015 – volume: 101 start-page: 898 year: 2008 ident: 10.1016/j.tins.2023.10.002_bb0205 article-title: Relation of locus coeruleus neurons in monkeys to Pavlovian and operant behaviors publication-title: J. Neurophysiol. doi: 10.1152/jn.91048.2008 – year: 2017 ident: 10.1016/j.tins.2023.10.002_bb0485 article-title: Count-based exploration with neural density models publication-title: Proceedings of the 34th International Conference on Machine Learning – volume: 71 start-page: 119 year: 2021 ident: 10.1016/j.tins.2023.10.002_bb0175 article-title: Orienting our view of the superior colliculus: specializations and general functions publication-title: Curr. Opin. Neurobiol. doi: 10.1016/j.conb.2021.10.005 – volume: 4 start-page: 442 year: 2021 ident: 10.1016/j.tins.2023.10.002_bb0545 article-title: Human belief state-based exploration and exploitation in an information-selective symmetric reversal bandit task publication-title: Comput. Brain Behav. doi: 10.1007/s42113-021-00112-3 – volume: 55 start-page: 268 year: 2015 ident: 10.1016/j.tins.2023.10.002_bb0090 article-title: Short- and long-lasting consequences of novelty, deviance and surprise on brain and cognition publication-title: Neurosci. Biobehav. Rev. doi: 10.1016/j.neubiorev.2015.05.002 – volume: 275 start-page: 1593 year: 1997 ident: 10.1016/j.tins.2023.10.002_bb0330 article-title: A neural substrate of prediction and reward publication-title: Science doi: 10.1126/science.275.5306.1593 – volume: 14 start-page: 1209 year: 2011 ident: 10.1016/j.tins.2023.10.002_bb0325 article-title: Lateral habenula neurons signal errors in the prediction of reward information publication-title: Nat. Neurosci. doi: 10.1038/nn.2902 – volume: 119 start-page: 8583 year: 2020 ident: 10.1016/j.tins.2023.10.002_bb0490 article-title: Planning to Explore via Self-Supervised World Models – volume: 146 start-page: 668 year: 1964 ident: 10.1016/j.tins.2023.10.002_bb0060 article-title: Visual experience in infants: decreased attention to familiar patterns relative to novel ones publication-title: Science doi: 10.1126/science.146.3644.668 – volume: 48 start-page: 162 year: 2011 ident: 10.1016/j.tins.2023.10.002_bb0170 article-title: The anatomical and functional relationship between the P3 and autonomic components of the orienting response publication-title: Psychophysiology doi: 10.1111/j.1469-8986.2010.01057.x – volume: 554 start-page: 244 year: 2018 ident: 10.1016/j.tins.2023.10.002_bb0275 article-title: Dopamine neuron activity before action initiation gates and invigorates future movements publication-title: Nature doi: 10.1038/nature25457 – volume: 4 start-page: 531 year: 2020 ident: 10.1016/j.tins.2023.10.002_bb0405 article-title: Shared striatal activity in decisions to satisfy curiosity and hunger at the risk of electric shocks publication-title: Nat. Hum. Behav. doi: 10.1038/s41562-020-0848-3 – volume: 74 start-page: 1309 year: 2008 ident: 10.1016/j.tins.2023.10.002_bb0575 article-title: An analysis of model-based interval estimation for Markov decision processes publication-title: J. Comput. Syst. Sci. doi: 10.1016/j.jcss.2007.08.009 – volume: 4 start-page: 907 year: 2013 ident: 10.1016/j.tins.2023.10.002_bb0625 article-title: Novelty or Surprise? publication-title: Front. Psychol. doi: 10.3389/fpsyg.2013.00907 – volume: 114 start-page: E3315 year: 2017 ident: 10.1016/j.tins.2023.10.002_bb0365 article-title: Parietal neurons encode expected gains in instrumental information publication-title: Proc. Natl. Acad. Sci. doi: 10.1073/pnas.1613844114 – volume: 23 start-page: 649 year: 2010 ident: 10.1016/j.tins.2023.10.002_bb0505 article-title: Of bits and wows: a Bayesian theory of surprise with applications to attention publication-title: Neural Netw. doi: 10.1016/j.neunet.2009.12.007 – volume: 19 start-page: 1294 year: 2009 ident: 10.1016/j.tins.2023.10.002_bb0065 article-title: Reward motivation accelerates the onset of neural novelty signals in humans to 85 milliseconds publication-title: Curr. Biol. CB doi: 10.1016/j.cub.2009.06.021 – volume: 108 start-page: 861 year: 2020 ident: 10.1016/j.tins.2023.10.002_bb0605 article-title: A platform for brain-wide volumetric functional ultrasound imaging and analysis of circuit dynamics in awake mice publication-title: Neuron doi: 10.1016/j.neuron.2020.09.020 – volume: 32 start-page: 2160 year: 2022 ident: 10.1016/j.tins.2023.10.002_bb0110 article-title: Surprise and recency in novelty detection in the primate brain publication-title: Curr. Biol. doi: 10.1016/j.cub.2022.03.064 – volume: 8 start-page: 1704 year: 2005 ident: 10.1016/j.tins.2023.10.002_bb0535 article-title: Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control publication-title: Nat. Neurosci. doi: 10.1038/nn1560 – volume: 17 year: 2021 ident: 10.1016/j.tins.2023.10.002_bb0390 article-title: Novelty is not surprise: human exploratory and adaptive behavior in sequential decision-making publication-title: PLoS Comput. Biol. doi: 10.1371/journal.pcbi.1009070 – volume: 2 start-page: 70 year: 2010 ident: 10.1016/j.tins.2023.10.002_bb0475 article-title: Intrinsically motivated reinforcement learning: an evolutionary perspective publication-title: IEEE Trans. Auton. Ment. Dev. doi: 10.1109/TAMD.2010.2051031 – volume: 12 start-page: 4942 year: 2021 ident: 10.1016/j.tins.2023.10.002_bb0540 article-title: Linear reinforcement learning in planning, grid fields, and cognitive control publication-title: Nat. Commun. doi: 10.1038/s41467-021-25123-3 – volume: 372 start-page: eabf4588 year: 2021 ident: 10.1016/j.tins.2023.10.002_bb0600 article-title: Neuropixels 2.0: a miniaturized high-density probe for stable, long-term brain recordings publication-title: Science doi: 10.1126/science.abf4588 – volume: 30 start-page: 1721 year: 2010 ident: 10.1016/j.tins.2023.10.002_bb0340 article-title: Contextual novelty changes reward representations in the striatum publication-title: J. Neurosci. doi: 10.1523/JNEUROSCI.5331-09.2010 – volume: 35 start-page: 31755 year: 2022 ident: 10.1016/j.tins.2023.10.002_bb0585 article-title: Modeling human exploration through resource-rational reinforcement learning publication-title: Advances in Neural Information Processing Systems – volume: 6 year: 2017 ident: 10.1016/j.tins.2023.10.002_bb0355 article-title: Opposite initialization to novel cues in dopamine signaling in ventral and posterior striatum in mice publication-title: eLife doi: 10.7554/eLife.21886 – volume: 13 start-page: 1526 year: 2010 ident: 10.1016/j.tins.2023.10.002_bb0215 article-title: Tuning arousal with optogenetic modulation of locus coeruleus neurons publication-title: Nat. Neurosci. doi: 10.1038/nn.2682 – volume: 129 start-page: 227 year: 2015 ident: 10.1016/j.tins.2023.10.002_bb0140 article-title: Perirhinal cortex lesions in rats: novelty detection and sensitivity to interference publication-title: Behav. Neurosci. doi: 10.1037/bne0000049 – volume: 67 start-page: 145 year: 1992 ident: 10.1016/j.tins.2023.10.002_bb0210 article-title: Responses of monkey dopamine neurons during learning of behavioral reactions publication-title: J. Neurophysiol. doi: 10.1152/jn.1992.67.1.145 – volume: 117 start-page: 15200 year: 2020 ident: 10.1016/j.tins.2023.10.002_bb0450 article-title: The ventral striatum dissociates information expectation, reward anticipation, and reward receipt publication-title: Proc. Natl. Acad. Sci. doi: 10.1073/pnas.1911778117 – volume: 56 start-page: 996 year: 1963 ident: 10.1016/j.tins.2023.10.002_bb0135 article-title: Role of auditory cortex in reflex head orientation by cats to auditory stimuli publication-title: J. Comp. Physiol. Psychol. doi: 10.1037/h0048640 – year: 2018 ident: 10.1016/j.tins.2023.10.002_bb0435 article-title: Computational theories of curiosity-driven learning publication-title: arXiv – volume: 34 start-page: 7947 year: 2014 ident: 10.1016/j.tins.2023.10.002_bb0370 article-title: Novelty enhances visual salience independently of reward in the parietal lobe publication-title: J. Neurosci. doi: 10.1523/JNEUROSCI.4171-13.2014 – volume: 25 start-page: 327 year: 2023 ident: 10.1016/j.tins.2023.10.002_bb0445 article-title: An information-theoretic perspective on intrinsic motivation in reinforcement learning: a survey publication-title: Entropy doi: 10.3390/e25020327 – volume: 38 start-page: 2579 year: 2018 ident: 10.1016/j.tins.2023.10.002_bb0385 article-title: Induction and relief of curiosity elicit parietal and frontal activity publication-title: J. Neurosci. doi: 10.1523/JNEUROSCI.2816-17.2018 – volume: 11 year: 2022 ident: 10.1016/j.tins.2023.10.002_bb0520 article-title: Independent and interacting value systems for reward and information in the human brain publication-title: eLife doi: 10.7554/eLife.66358 – volume: 34 start-page: 24379 year: 2021 ident: 10.1016/j.tins.2023.10.002_bb0495 article-title: Discovering and achieving goals via world models publication-title: Adv. Neural Inf. Proces. Syst. – volume: 225 year: 2022 ident: 10.1016/j.tins.2023.10.002_bb0510 article-title: Contributions of expected learning progress and perceptual novelty to curiosity-driven exploration publication-title: Cognition doi: 10.1016/j.cognition.2022.105119 – volume: 25 start-page: 545 year: 1963 ident: 10.1016/j.tins.2023.10.002_bb0045 article-title: Higher nervous functions: the orienting reflex publication-title: Annu. Rev. Physiol. doi: 10.1146/annurev.ph.25.030163.002553 – volume: 42 start-page: 3117 year: 2015 ident: 10.1016/j.tins.2023.10.002_bb0145 article-title: Perirhinal cortex lesions impair tests of object recognition memory but spare novelty detection publication-title: Eur. J. Neurosci. doi: 10.1111/ejn.13106 – volume: 776 start-page: 61 year: 1997 ident: 10.1016/j.tins.2023.10.002_bb0250 article-title: Regional and temporal differences in real-time dopamine efflux in the nucleus accumbens during free-choice novelty publication-title: Brain Res. doi: 10.1016/S0006-8993(97)01004-4 – volume: 3 start-page: 587 year: 2019 ident: 10.1016/j.tins.2023.10.002_bb0305 article-title: Diverse motives for human curiosity publication-title: Nat. Hum. Behav. doi: 10.1038/s41562-019-0589-3 – volume: 20 start-page: 451 year: 2000 ident: 10.1016/j.tins.2023.10.002_bb0165 article-title: Impaired recognition memory in monkeys after damage limited to the hippocampal region publication-title: J. Neurosci. doi: 10.1523/JNEUROSCI.20-01-00451.2000 – volume: 2 start-page: 181 year: 1995 ident: 10.1016/j.tins.2023.10.002_bb0230 article-title: Novelty seeking behavior in the rat is dependent upon the integrity of the noradrenergic system publication-title: Cogn. Brain Res. doi: 10.1016/0926-6410(95)90007-1 – volume: 84 start-page: 486 year: 2014 ident: 10.1016/j.tins.2023.10.002_bb0310 article-title: States of curiosity modulate hippocampus-dependent learning via the dopaminergic circuit publication-title: Neuron doi: 10.1016/j.neuron.2014.08.060 – volume: 82 year: 2023 ident: 10.1016/j.tins.2023.10.002_bb0085 article-title: Surprise and novelty in the brain publication-title: Curr. Opin. Neurobiol. doi: 10.1016/j.conb.2023.102758 – volume: 69 start-page: 1918 year: 1993 ident: 10.1016/j.tins.2023.10.002_bb0100 article-title: The representation of stimulus familiarity in anterior inferior temporal cortex publication-title: J. Neurophysiol. doi: 10.1152/jn.1993.69.6.1918 – volume: 106 start-page: 142 year: 2020 ident: 10.1016/j.tins.2023.10.002_bb0245 article-title: Cue-evoked dopamine promotes conditioned responding during learning publication-title: Neuron doi: 10.1016/j.neuron.2020.01.012 – volume: 93 start-page: 367 year: 1993 ident: 10.1016/j.tins.2023.10.002_bb0130 article-title: The effects of stimulus novelty and familiarity on neuronal activity in the amygdala of monkeys performing recognition memory tasks publication-title: Exp. Brain Res. doi: 10.1007/BF00229353 – volume: 66 start-page: 81 year: 1995 ident: 10.1016/j.tins.2023.10.002_bb0195 article-title: Increases in hippocampal and frontal cortical acetylcholine release associated with presentation of sensory stimuli publication-title: Neuroscience doi: 10.1016/0306-4522(94)00578-S – volume: 10 start-page: 5168 year: 2019 ident: 10.1016/j.tins.2023.10.002_bb0360 article-title: A neural network for information seeking publication-title: Nat. Commun. doi: 10.1038/s41467-019-13135-z – volume: 19 start-page: 758 year: 2018 ident: 10.1016/j.tins.2023.10.002_bb0075 article-title: Towards a neuroscience of active sampling and curiosity publication-title: Nat. Rev. Neurosci. doi: 10.1038/s41583-018-0078-0 – volume: 22 start-page: 1327 year: 2019 ident: 10.1016/j.tins.2023.10.002_bb0375 article-title: Parietal neurons encode information sampling based on decision uncertainty publication-title: Nat. Neurosci. doi: 10.1038/s41593-019-0440-1 – volume: 28 start-page: 2125 year: 2015 ident: 10.1016/j.tins.2023.10.002_bb0570 article-title: Variational information maximisation for intrinsically motivated reinforcement learning publication-title: Adv. Neural Inf. Proces. Syst. – volume: 7 start-page: 1180 year: 1995 ident: 10.1016/j.tins.2023.10.002_bb0220 article-title: Response to novelty and its rapid habituation in locus coeruleus neurons of the freely exploring rat publication-title: Eur. J. Neurosci. doi: 10.1111/j.1460-9568.1995.tb01108.x – volume: 85 start-page: 602 year: 2015 ident: 10.1016/j.tins.2023.10.002_bb0410 article-title: Orbitofrontal cortex uses distinct codes for different choice attributes in decisions motivated by curiosity publication-title: Neuron doi: 10.1016/j.neuron.2014.12.050 – year: 2018 ident: 10.1016/j.tins.2023.10.002_bb0470 – volume: 51 start-page: 369 year: 2006 ident: 10.1016/j.tins.2023.10.002_bb0255 article-title: Absolute coding of stimulus novelty in the human substantia nigra/VTA publication-title: Neuron doi: 10.1016/j.neuron.2006.06.021 – volume: 7 start-page: 1481 year: 2023 ident: 10.1016/j.tins.2023.10.002_bb0555 article-title: Empowerment contributes to exploration behaviour in a creative video game publication-title: Nat. Hum. Behav. doi: 10.1038/s41562-023-01661-2 – volume: 22 start-page: 243 year: 2012 ident: 10.1016/j.tins.2023.10.002_bb0050 article-title: Sniffing and whisking in rodents publication-title: Curr. Opin. Neurobiol. doi: 10.1016/j.conb.2011.11.013 – volume: 110 start-page: 3789 year: 2022 ident: 10.1016/j.tins.2023.10.002_bb0280 article-title: Striatal dopamine explains novelty-induced behavioral dynamics and individual variability in threat prediction publication-title: Neuron doi: 10.1016/j.neuron.2022.08.022 – volume: 45 start-page: 180 year: 1954 ident: 10.1016/j.tins.2023.10.002_bb0020 article-title: A theory of human curiosity publication-title: Br. J. Psychol. Gen. Sect. – volume: 25 start-page: 50 year: 2022 ident: 10.1016/j.tins.2023.10.002_bb0290 article-title: A primate temporal cortex–zona incerta pathway for novelty seeking publication-title: Nat. Neurosci. doi: 10.1038/s41593-021-00950-1 – volume: 101 start-page: 5064 year: 2004 ident: 10.1016/j.tins.2023.10.002_bb0160 article-title: Visual memory task for rats reveals an essential role for hippocampus and perirhinal cortex publication-title: Proc. Natl. Acad. Sci. doi: 10.1073/pnas.0308528101 – volume: 96 start-page: 457 year: 1993 ident: 10.1016/j.tins.2023.10.002_bb0115 article-title: Neuronal activity related to visual recognition memory: long-term memory and the encoding of recency and familiarity information in the primate anterior and medial inferior temporal and rhinal cortex publication-title: Exp. Brain Res. doi: 10.1007/BF00234113 – volume: 8 start-page: 2148 year: 2017 ident: 10.1016/j.tins.2023.10.002_bb0095 article-title: Neurons along the auditory pathway exhibit a hierarchical organization of prediction error publication-title: Nat. Commun. doi: 10.1038/s41467-017-02038-6 – volume: 58 start-page: 167 year: 2019 ident: 10.1016/j.tins.2023.10.002_bb0630 article-title: Visual novelty, curiosity, and intrinsic reward in machine learning and the brain publication-title: Curr. Opin. Neurobiol. doi: 10.1016/j.conb.2019.08.004 – year: 2020 ident: 10.1016/j.tins.2023.10.002_bb0595 article-title: Meta-learning curiosity algorithms publication-title: International Conference on Learning Representations – volume: 49 start-page: 1295 year: 2009 ident: 10.1016/j.tins.2023.10.002_bb0500 article-title: Bayesian surprise attracts human attention publication-title: Vis. Res. doi: 10.1016/j.visres.2008.09.007 – volume: 49 start-page: 1254 year: 1983 ident: 10.1016/j.tins.2023.10.002_bb0180 article-title: Visual and oculomotor functions of monkey substantia nigra pars reticulata. II. Visual responses related to fixation of gaze publication-title: J. Neurophysiol. doi: 10.1152/jn.1983.49.5.1254 – volume: 55 start-page: 7 year: 2019 ident: 10.1016/j.tins.2023.10.002_bb0425 article-title: The algorithmic architecture of exploration in the human brain publication-title: Curr. Opin. Neurobiol. doi: 10.1016/j.conb.2018.11.003 – volume: 97 start-page: 5062 year: 2019 ident: 10.1016/j.tins.2023.10.002_bb0565 article-title: Self-supervised exploration via disagreement – year: 2015 ident: 10.1016/j.tins.2023.10.002_bb0010 – volume: 20 start-page: 963 year: 2009 ident: 10.1016/j.tins.2023.10.002_bb0350 article-title: The wick in the candle of learning: epistemic curiosity activates reward circuitry and enhances memory publication-title: Psychol. Sci. doi: 10.1111/j.1467-9280.2009.02402.x – volume: 193 start-page: 1142 year: 1976 ident: 10.1016/j.tins.2023.10.002_bb0620 article-title: The effect of stimulus sequence on the waveform of the cortical event-related potential publication-title: Science doi: 10.1126/science.959831 – volume: 11 start-page: 794 year: 2004 ident: 10.1016/j.tins.2023.10.002_bb0155 article-title: Impaired odor recognition memory in patients with hippocampal lesions publication-title: Learn. Mem. doi: 10.1101/lm.82504 – volume: 537 start-page: 357 year: 2016 ident: 10.1016/j.tins.2023.10.002_bb0225 article-title: Locus coeruleus and dopaminergic consolidation of everyday memory publication-title: Nature doi: 10.1038/nature19325 – volume: 127 start-page: 455 year: 2019 ident: 10.1016/j.tins.2023.10.002_bb0580 article-title: Reconciling novelty and complexity through a rational analysis of curiosity publication-title: Psychol. Rev. doi: 10.1037/rev0000175 – volume: 15 start-page: 549 year: 2002 ident: 10.1016/j.tins.2023.10.002_bb0515 article-title: Dopamine: generalization and bonuses publication-title: Neural Netw. doi: 10.1016/S0893-6080(02)00048-5 – volume: 1 start-page: 128 year: 2005 ident: 10.1016/j.tins.2023.10.002_bb0560 article-title: Empowerment: a universal agent-centric measure of control
SSID	ssj0014814
Score	2.5631998
SecondaryResourceType	review_article
Snippet	Exploratory behaviors are central to the study of curiosity, since they allow us to infer an otherwise hidden state of being curious from quantitative,... Curiosity refers to the intrinsic desire of humans and animals to explore the unknown, even when there is no apparent reason to do so. Thus far, no single,...
SourceID	proquest pubmed crossref elsevier
SourceType	Aggregation Database Index Database Enrichment Source Publisher
StartPage	1054
SubjectTerms	Animals Computer Simulation Exploratory Behavior - physiology Humans information Motivation neural circuits neuromodulatory systems Neurosciences novelty orienting reinforcement learning surprise
Title	Curiosity-driven exploration: foundations in neuroscience and computational modeling
URI	https://www.clinicalkey.com/#!/content/1-s2.0-S0166223623002400 https://www.ncbi.nlm.nih.gov/pubmed/37925342 https://www.proquest.com/docview/2886329167
Volume	46
WOSCitedRecordID	wos001123389800001&url=https%3A%2F%2Fcvtisr.summon.serialssolutions.com%2F%23%21%2Fsearch%3Fho%3Df%26include.ft.matches%3Dt%26l%3Dnull%26q%3D
hasFullText	1
inHoldings	1
isFullTextHit
isPrint
journalDatabaseRights	– providerCode: PRVESC databaseName: Elsevier SD Freedom Collection Journals 2021 customDbUrl: eissn: 1878-108X dateEnd: 99991231 omitProxy: false ssIdentifier: ssj0014814 issn: 0166-2236 databaseCode: AIEXJ dateStart: 19950101 isFulltext: true titleUrlDefault: https://www.sciencedirect.com providerName: Elsevier
link	http://cvtisr.summon.serialssolutions.com/2.0.0/link/0/eLvHCXMwtV1bb9MwFLa6DiFeEGxcymUyEtpLlSlxLnZ4q6aOW1cQdNC3KHUc0apLQ5qObb-In8mxY6eh2qbxwEsUWXEb-Xw5F_s75yD02iM2dxLQfoJx1_ICEcMnFQKWSco5mQg7USzfbwM6HLLxOPzcav02uTBnc5pl7Pw8zP-rqGEMhC1TZ_9B3PWPwgDcg9DhCmKH660Ef7gqppKJdWElhVRlsoj_XAtaxv9p3UhJUWEbBS2FSXLLV6XZI1Sdcox5m5UbPNrmZLV9-0W1ti-6Ijlo7DEcL8ByLn_EmeCqh3C3NwdNe1lbhI-LLCnAlRW_pvyy4njwfE0bfitdVJ2Y8301T4v4dK04xVLnMn4VYJFLg3a9kUHcBimk0r1MBrS26jQMpumKMa2w9Z6lBiZpqF9wFr0r7UK1RTEDNyCTNdqJe6AofWRtBc3J__BTdHQyGESj_ni0n_-0ZH8yeY6vm7VsoW1C_ZC10XbvfX_8oT6x8piqI1-_sE7QqriEm397nRN0XZCjnJ3RA3RfRym4V6HrIWqJbAft9rK4XJxe4H2seMPqQGYH3T3W9IxdNNrEHm5g7w1uIA9PM9wEDwbk4b-Qhw3yHqGTo_7o8J2l23ZY3POd0vIEC2OPJDYV1I9p7AgwCw6b2Nx1UhJLg8pCTl0aJr4_YbKmAUS5nLOUJEFgp-5j1M4WmXiKsM3TNJUdExzieNKX9yEACJLUp1Q4LPA7yDHrGHFd0162VplHhrw4i-TaR3Lt5RisfQd16zl5VdHlxqddI57I5CqDdY0AWDfOemUkGYG6lmdw8IktVvAQY4FLICajHfSkEnH9FrAixHc98uwWs5-je-tv6AVql8VKvER3-Fk5XRZ7aIuO2Z6G6B-6N8Rh
linkProvider	Elsevier
openUrl	ctx_ver=Z39.88-2004&ctx_enc=info%3Aofi%2Fenc%3AUTF-8&rfr_id=info%3Asid%2Fsummon.serialssolutions.com&rft_val_fmt=info%3Aofi%2Ffmt%3Akev%3Amtx%3Ajournal&rft.genre=article&rft.atitle=Curiosity-driven+exploration%3A+foundations+in+neuroscience+and+computational+modeling&rft.jtitle=Trends+in+neurosciences+%28Regular+ed.%29&rft.au=Modirshanechi%2C+Alireza&rft.au=Kondrakiewicz%2C+Kacper&rft.au=Gerstner%2C+Wulfram&rft.au=Haesler%2C+Sebastian&rft.date=2023-12-01&rft.issn=1878-108X&rft.eissn=1878-108X&rft.volume=46&rft.issue=12&rft.spage=1054&rft_id=info:doi/10.1016%2Fj.tins.2023.10.002&rft.externalDBID=NO_FULL_TEXT
thumbnail_l	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/lc.gif&issn=0166-2236&client=summon
thumbnail_m	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/mc.gif&issn=0166-2236&client=summon
thumbnail_s	http://covers-cdn.summon.serialssolutions.com/index.aspx?isbn=/sc.gif&issn=0166-2236&client=summon