Foundations of human reasoning in the prefrontal cortex

Selecting the most successful strategy The brain's prefrontal cortex helps us to make decisions in an uncertain and constantly changing environment. Donoso et al. present a model of human reasoning as an algorithm implemented in the prefrontal cortex (see the Perspective by Hare). Brain-imaging experiments supported this model. Depending on the prevailing circumstances, human reasoning can either adapt ongoing behavioral strategies or switch to previously learned strategies. Only when neither approach is appropriate will the brain create new strategies. Science, this issue p. 1481, see also p. 1446 Human reasoning combines inferential and creative processes. [Also see Perspective by Hare] The prefrontal cortex (PFC) subserves reasoning in the service of adaptive behavior. Little is known, however, about the architecture of reasoning processes in the PFC. Using computational modeling and neuroimaging, we show here that the human PFC has two concurrent inferential tracks: (i) one from ventromedial to dorsomedial PFC regions that makes probabilistic inferences about the reliability of the ongoing behavioral strategy and arbitrates between adjusting this strategy versus exploring new ones from long-term memory, and (ii) another from polar to lateral PFC regions that makes probabilistic inferences about the reliability of two or three alternative strategies and arbitrates between exploring new strategies versus exploiting these alternative ones. The two tracks interact and, along with the striatum, realize hypothesis testing for accepting versus rejecting newly created strategies.

[1]  E. Koechlin,et al.  The role of the anterior prefrontal cortex in human cognition , 1999, Nature.

[2]  Joshua W. Brown,et al.  Medial prefrontal cortex as an action-outcome predictor , 2011, Nature Neuroscience.

[3]  D. Blei,et al.  Context, learning, and extinction. , 2010, Psychological review.

[4]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[5]  R. Rosenfeld.  Nature , 2009, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[6]  Mitsuo Kawato,et al.  Multiple Model-Based Reinforcement Learning , 2002, Neural Computation.

[7]  K. Semendeferi,et al.  Human prefrontal cortex: evolution, development, and pathology. , 2012, Progress in brain research.

[8]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[9]  Finale Doshi-Velez,et al.  The Infinite Partially Observable Markov Decision Process , 2009, NIPS.

[10]  Kristina M. Visscher,et al.  A Core System for the Implementation of Task Sets , 2006, Neuron.

[11]  Aaron C. Courville,et al.  The pigeon as particle filter , 2007, NIPS 2007.

[12]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[13]  E. Koechlin,et al.  Anterior Prefrontal Function and the Limits of Human Decision-Making , 2007, Science.

[14]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[15]  E. Koechlin,et al.  The Architecture of Cognitive Control in the Human Prefrontal Cortex , 2003, Science.

[16]  Timothy E. J. Behrens,et al.  Counterfactual Choice and Learning in a Neural Network Centered on Human Lateral Frontopolar Cortex , 2011, PLoS biology.

[17]  R. Passingham,et al.  Prefrontal interactions reflect future task operations , 2003, Nature Neuroscience.

[18]  Benjamin J. Tamber-Rosenau,et al.  Avoiding non-independence in fMRI data analysis: Leave one subject out , 2010, NeuroImage.

[19]  Adam G. Thomas,et al.  Comparison of Human Ventral Frontal Cortex Areas for Cognitive Control and Language with Areas in Monkey Frontal Cortex , 2014, Neuron.

[20]  C. Summerfield,et al.  An information theoretical approach to prefrontal executive function , 2007, Trends in Cognitive Sciences.

[21]  E. Koechlin,et al.  Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making , 2012, PLoS biology.

[22]  R. Dolan,et al.  Confidence in value-based choice , 2012, Nature Neuroscience.

[23]  D. Wilkin,et al.  Neuron , 2001, Brain Research.

[24]  Michael I. Jordan,et al.  Hierarchical Dirichlet Processes , 2006 .

[25]  Etienne Koechlin,et al.  Divided Representation of Concurrent Goals in the Human Frontal Lobes , 2010, Science.

[26]  G. E. Alexander,et al.  Parallel organization of functionally segregated circuits linking basal ganglia and cortex. , 1986, Annual review of neuroscience.

[27]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[28]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[29]  M. D’Esposito,et al.  Frontal Cortex and the Discovery of Abstract Action Rules , 2010, Neuron.

[30]  K. Sakai.  Task set and prefrontal cortex. , 2008, Annual review of neuroscience.

[31]  Kenji Doya,et al.  Reinforcement learning: Computational theory and biological mechanisms , 2007, HFSP journal.

[32]  Joseph T. McGuire,et al.  A Neural Signature of Hierarchical Reinforcement Learning , 2011, Neuron.

[33]  Timothy E. J. Behrens,et al.  Neural Mechanisms of Foraging , 2012, Science.

[34]  N. Chater,et al.  Précis of Bayesian Rationality: The Probabilistic Approach to Human Reasoning , 2009, Behavioral and Brain Sciences.

[35]  Timothy Edward John Behrens,et al.  How Green Is the Grass on the Other Side? Frontopolar Cortex and the Evidence in Favor of Alternative Courses of Action , 2009, Neuron.