Neural correlates of strategic reasoning during competitive games

Although human and animal behaviors are largely shaped by reinforcement and punishment, choices in social settings are also influenced by information about the knowledge and experience of other decision-makers. During competitive games, monkeys increased their payoffs by systematically deviating from a simple heuristic learning algorithm and thereby countering the predictable exploitation by their computer opponent. Neurons in the dorsomedial prefrontal cortex (dmPFC) signaled the animal’s recent choice and reward history that reflected the computer’s exploitative strategy. The strength of switching signals in the dmPFC also correlated with the animal’s tendency to deviate from the heuristic learning algorithm. Therefore, the dmPFC might provide control signals for overriding simple heuristic learning algorithms based on the inferred strategies of the opponent. Neuronal responses in the dorsomedial prefrontal cortex predict choices and switches in gaming strategies in monkeys. Smart monkeys can outwit a computer What happens in the brain when we are learning to compete against an opponent? Seo et al. observed monkeys competing against a computer that can adapt to the monkey's behavior. The monkeys switched their learning strategies when they worked out that their opponent was reacting to their behavior. The responses of the dorsomedial prefrontal cortex cells in the monkey brains predicted their choices and switches in strategies. Science, this issue p. 340

[1]  J. Nash Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[2]  Colin Camerer Behavioral Game Theory , 1990 .

[3]  J Tanji,et al.  Role for cells in the presupplementary motor area in updating motor plans. , 1996, Proceedings of the National Academy of Sciences of the United States of America.

[4]  Trevor Hastie,et al.  The Elements of Statistical Learning , 2001 .

[5]  C. Frith,et al.  Functional imaging of ‘theory of mind’ , 2003, Trends in Cognitive Sciences.

[6]  D. Barraclough,et al.  Prefrontal cortex and decision making in a mixed-strategy game , 2004, Nature Neuroscience.

[7]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[8]  Aldo Genovesio,et al.  Representation of Future and Previous Spatial Goals by Separate Neural Populations in Prefrontal Cortex , 2006, The Journal of Neuroscience.

[9]  O. Hikosaka,et al.  Switching from automatic to controlled action by monkey medial frontal cortex , 2007, Nature Neuroscience.

[10]  H. Seo,et al.  Temporal Filtering of Reward Signals in the Dorsal Anterior Cingulate Cortex during a Mixed-Strategy Game , 2007, The Journal of Neuroscience.

[11]  Peter Bossaerts,et al.  Neural correlates of mentalizing-related computations during strategic interactions in humans , 2008, Proceedings of the National Academy of Sciences.

[12]  Gerd Gigerenzer,et al.  Homo Heuristicus: Why Biased Minds Make Better Inferences , 2009, Top. Cogn. Sci..

[13]  Daeyeol Lee,et al.  Behavioral and Neural Changes after Gains and Losses of Conditioned Reinforcers , 2009, The Journal of Neuroscience.

[14]  Matthew F S Rushworth,et al.  The Computation of Social Behavior , 2009, Science.

[15]  H. Seo,et al.  Lateral Intraparietal Cortex and Reinforcement Learning during a Mixed-Strategy Game , 2009, Journal of Neuroscience.

[16]  O. Hikosaka,et al.  Perceptual Learning, Motor Learning and Automaticity Switching from Automatic to Controlled Behavior: Cortico-basal Ganglia Mechanisms , 2022 .

[17]  Jung Hoon Sul,et al.  Distinct Roles of Rodent Orbitofrontal and Medial Prefrontal Cortex in Decision Making , 2010, Neuron.

[18]  Aldo Genovesio,et al.  Evaluating self-generated decisions in frontal pole cortex of monkeys , 2009, Nature Neuroscience.

[19]  Daeyeol Lee,et al.  Ubiquity and Specificity of Reinforcement Signals throughout the Human Brain , 2011, Neuron.

[20]  H. Seo,et al.  A reservoir of time constants for memory traces in cortical neurons , 2011, Nature Neuroscience.

[21]  M. Frank,et al.  From reinforcement learning models to psychiatric and neurological disorders , 2011, Nature Neuroscience.

[22]  K. Doya,et al.  Multiple representations and algorithms for reinforcement learning in the cortico-basal ganglia circuit , 2011, Current Opinion in Neurobiology.

[23]  Justin L. Gardner,et al.  Learning to Simulate Others' Decisions , 2012, Neuron.

[24]  H. Seo,et al.  Neural basis of reinforcement learning and decision making. , 2012, Annual review of neuroscience.

[25]  M. Botvinick Hierarchical reinforcement learning and decision making , 2012, Current Opinion in Neurobiology.

[26]  Kyle E. Mathewson,et al.  Dissociable neural representations of reinforcement and belief prediction errors underlie strategic learning , 2012, Proceedings of the National Academy of Sciences.

[27]  Daeyeol Lee,et al.  Signals for Previous Goal Choice Persist in the Dorsomedial, but Not Dorsolateral Striatum of Rats , 2013, The Journal of Neuroscience.

[28]  Daeyeol Lee,et al.  Cortical Signals for Rewarded Actions and Strategic Exploration , 2013, Neuron.

[29]  Daeyeol Lee Decision Making: From Neuroscience to Psychiatry , 2013, Neuron.

[30]  Shinsuke Shimojo,et al.  Neural Computations Underlying Arbitration between Model-Based and Model-free Learning , 2013, Neuron.