Explore/Exploit: Tradeoffs in Decision Making

Abstract

Explore-exploit tradeoffs–the conflict between exploring novel options and exploiting familiar ones–is a fundamental decision model adapted from basic and translational science. Striking the right balance between these two strategies is pivotal for achieving efficient outcomes and adapting to varying levels of uncertainty. Individuals must also adapt to varying levels of confidence and external factors that hold implications for their decisions. This review aims to shed light on the influential role of various cognitive (e.g., confidence, bias) and affective processes (e.g., stress) on explore-exploit decision making. We also cover the role modern neuroscience has played in studying this tradeoff and its underlying neural circuitry. This topic holds profound importance in making real-world developments across diverse disciplines. In economics, understanding how confidence impacts decision making can elucidate market behaviors and financial choices. In addition, this research advances models of artificial intelligence and human-computer interaction (HCI), which are highly reliant on understanding principles of decision. Lastly, understanding the underlying brain pathways can provide psychological insights into cognitive flexibility, motivational tendencies, and human learning; indeed, these are critical processes that, if perturbed, underscore the etiology and maintenance of a variety of psychiatric illnesses.