Humans use directed and random exploration to solve the explore-exploit dilemma

All adaptive organisms face the fundamental tradeoff between pursuing a known reward (exploitation) and sampling lesser-known options in search of something better (exploration). Theory suggests at least two strategies for solving this dilemma: a directed strategy in which choices are explicitly bia...

Full description

Saved in:
Bibliographic Details
Published in:Journal of experimental psychology. General Vol. 143; no. 6; p. 2074
Main Authors: Wilson, Robert C, Geana, Andra, White, John M, Ludvig, Elliot A, Cohen, Jonathan D
Format: Journal Article
Language:English
Published: United States 01.12.2014
Subjects:
Adolescent
Choice Behavior - physiology
Decision Making - physiology
Exploratory Behavior - physiology
Female
Humans
Male
Neuropsychological Tests
Reward
Young Adult
ISSN:1939-2222, 1939-2222
Online Access:Get more information
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:All adaptive organisms face the fundamental tradeoff between pursuing a known reward (exploitation) and sampling lesser-known options in search of something better (exploration). Theory suggests at least two strategies for solving this dilemma: a directed strategy in which choices are explicitly biased toward information seeking, and a random strategy in which decision noise leads to exploration by chance. In this work we investigated the extent to which humans use these two strategies. In our "Horizon task," participants made explore-exploit decisions in two contexts that differed in the number of choices that they would make in the future (the time horizon). Participants were allowed to make either a single choice in each game (horizon 1), or 6 sequential choices (horizon 6), giving them more opportunity to explore. By modeling the behavior in these two conditions, we were able to measure exploration-related changes in decision making and quantify the contributions of the two strategies to behavior. We found that participants were more information seeking and had higher decision noise with the longer horizon, suggesting that humans use both strategies to solve the exploration-exploitation dilemma. We thus conclude that both information seeking and choice variability can be controlled and put to use in the service of exploration.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:1939-2222
1939-2222
DOI:10.1037/a0038199