Dopamine reward prediction errors reflect hidden-state inference across time

A long-standing idea in modern neuroscience is that the brain computes inferences about the outside world rather than passively observing its environment. The authors record from midbrain dopamine neurons during tasks with different reward contingencies and show that responses are consistent with a...

Ausführliche Beschreibung

Gespeichert in:

Bibliographische Detailangaben
Veröffentlicht in:	Nature neuroscience Jg. 20; H. 4; S. 581 - 589
Hauptverfasser:	Starkweather, Clara Kwon, Babayan, Benedicte M, Uchida, Naoshige, Gershman, Samuel J
Format:	Journal Article
Sprache:	Englisch
Veröffentlicht:	New York Nature Publishing Group US 01.04.2017 Nature Publishing Group
Schlagworte:	631/378/116/2396 631/378/1788 9/97 Animal Genetics and Genomics Animals Association Learning - physiology Behavioral Sciences Biological Techniques Biomedicine Dopamine Dopaminergic Neurons - physiology Health aspects Inference Male Mice Models, Neurological Neurobiology Neurosciences Normal distribution Odors Physiological aspects Psychological aspects Recording sessions Reward Rewards (Psychology) Time Factors Ventral Tegmental Area - physiology United States United States > US Massachusetts
ISSN:	1097-6256, 1546-1726, 1546-1726
Online-Zugang:	Volltext
Tags:	Tag hinzufügen Keine Tags, Fügen Sie den ersten Tag hinzu!

Beschreibung
Zusammenfassung:	A long-standing idea in modern neuroscience is that the brain computes inferences about the outside world rather than passively observing its environment. The authors record from midbrain dopamine neurons during tasks with different reward contingencies and show that responses are consistent with a learning rule that harnesses hidden-state inference. Midbrain dopamine neurons signal reward prediction error (RPE), or actual minus expected reward. The temporal difference (TD) learning model has been a cornerstone in understanding how dopamine RPEs could drive associative learning. Classically, TD learning imparts value to features that serially track elapsed time relative to observable stimuli. In the real world, however, sensory stimuli provide ambiguous information about the hidden state of the environment, leading to the proposal that TD learning might instead compute a value signal based on an inferred distribution of hidden states (a 'belief state'). Here we asked whether dopaminergic signaling supports a TD learning framework that operates over hidden states. We found that dopamine signaling showed a notable difference between two tasks that differed only with respect to whether reward was delivered in a deterministic manner. Our results favor an associative learning rule that combines cached values with hidden-state inference.
Bibliographie:	ObjectType-Article-1 SourceType-Scholarly Journals-1 ObjectType-Feature-2 content type line 14 content type line 23
ISSN:	1097-6256 1546-1726 1546-1726
DOI:	10.1038/nn.4520