A novel fMRI paradigm to dissociate the behavioral and neural components of mixed‐strategy decision making from non‐strategic decisions in humans

During competitive interactions, such as predator–prey or team sports, the outcome of one's actions is dependent on both their own choices and those of their opponents. Success in these rivalries requires that individuals choose dynamically and unpredictably, often adopting a mixed strategy. Understanding the neural basis of strategic decision making is complicated by the fact that it recruits various cognitive processes that are often shared with non‐strategic forms of decision making, such as value estimation, working memory, response inhibition, response selection, and reward processes. Although researchers have explored neural activity within key brain regions during mixed‐strategy games, how brain activity differs in the context of strategic interactions versus non‐strategic choices is not well understood. We developed a novel behavioral paradigm to dissociate choice behavior during mixed‐strategy interactions from non‐strategic choices, and we used task‐based functional magnetic resonance imaging (fMRI) to contrast brain activation. In a block design, participants competed in the classic mixed‐strategy game, “matching pennies,” against a dynamic computer opponent designed to exploit predictability in players’ response patterns. Results were contrasted with a non‐strategic task that had comparable sensory input, motor output, and reward rate; thus, differences in behavior and brain activation reflect strategic processes. The mixed‐strategy game was associated with activation of a distributed cortico‐striatal network compared to the non‐strategic task. We propose that choosing in mixed‐strategy contexts requires additional cognitive demands present to a lesser degree during the control task, illustrating the strength of this design in probing function of cognitive systems beyond core sensory, motor, and reward processes.

[1]  E. Rowland Theory of Games and Economic Behavior , 1946, Nature.

[2]  E. Fehr,et al.  The neurobiology of rewards and values in social decision making , 2014, Nature Reviews Neuroscience.

[3]  Michael J. Frank,et al.  A dopaminergic basis for working memory, learning and attentional shifting in Parkinsonism , 2008, Neuropsychologia.

[4]  Katia M. Harlé,et al.  Social economic decision-making across the lifespan: An fMRI investigation , 2012, Neuropsychologia.

[5]  Jonathan D. Cohen,et al.  The Neural Basis of Economic Decision-Making in the Ultimatum Game , 2003, Science.

[6]  K. Doya,et al.  Representation of Action-Specific Reward Values in the Striatum , 2005, Science.

[7]  M. Dorris,et al.  Role of the Superior Colliculus in Choosing Mixed-Strategy Saccades , 2009, The Journal of Neuroscience.

[8]  V. Menon,et al.  Saliency, switching, attention and control: a network model of insula function , 2010, Brain Structure and Function.

[9]  Martin P. Paulus,et al.  Superior temporal gyrus and insula provide response and outcome-dependent information during assessment and action selection in a decision-making situation , 2005, NeuroImage.

[10]  Daeyeol Lee Decision Making: From Neuroscience to Psychiatry , 2013, Neuron.

[11]  M. Delgado,et al.  Perceptions of moral character modulate the neural systems of reward during the trust game , 2005, Nature Neuroscience.

[12]  S. Yamada,et al.  Mind-Reading and Behavior-Reading against Agents with and without Anthropomorphic Features in a Competitive Situation , 2017, Front. Psychol..

[13]  Michael X. Cohen,et al.  Behavioral / Systems / Cognitive Reinforcement Learning Signals Predict Future Decisions , 2007 .

[14]  M. Gluck,et al.  Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients. , 2009, Brain : a journal of neurology.

[15]  K. Cheng,et al.  Neural basis of decision making guided by emotional outcomes. , 2015, Journal of neurophysiology.

[16]  S. Huettel,et al.  A Distinct Role of the Temporal-Parietal Junction in Predicting Socially Guided Decisions , 2012, Science.

[17]  D. Munoz,et al.  Saccadic Probability Influences Motor Preparation Signals and Time to Saccadic Initiation , 1998, The Journal of Neuroscience.

[18]  C. H. Donahue,et al.  Neural correlates of strategic reasoning during competitive games , 2014, Science.

[19]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[20]  S. Lewis,et al.  The role of learned irrelevance in attentional set-shifting impairments in Parkinson's disease. , 2006, Neuropsychology.

[21]  Colin Camerer,et al.  A framework for studying the neurobiology of value-based decision making , 2008, Nature Reviews Neuroscience.

[22]  M. Frank,et al.  From reinforcement learning models to psychiatric and neurological disorders , 2011, Nature Neuroscience.

[23]  Mukesh Dhamala,et al.  Hyperscanning : Simultaneous fMRI during Linked Social Interactions , 2001 .

[24]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[25]  P. Glimcher,et al.  Action and Outcome Encoding in the Primate Caudate Nucleus , 2007, The Journal of Neuroscience.

[26]  M. Gluck,et al.  l-dopa impairs learning, but spares generalization, in Parkinson's disease , 2006, Neuropsychologia.

[27]  D. Barraclough,et al.  Reinforcement learning and decision making in monkeys during a competitive game. , 2004, Brain research. Cognitive brain research.

[28]  Guy B. Williams,et al.  Carrots and sticks fail to change behavior in cocaine addiction , 2016, Science.

[29]  M. Dorris,et al.  Temporal and spatial allocation of motor preparation during a mixed-strategy game. , 2008, Journal of neurophysiology.

[30]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[31]  A. Sanfey,et al.  Games in Humans and Non-human Primates: Scanners to Single Units , 2009 .

[32]  C. Kennard,et al.  The role of the pre-supplementary motor area in the control of action , 2007, NeuroImage.

[33]  R. Thaler Behavioral Economics: Past, Present and Future , 2016 .

[34]  M. Frank Computational models of motivated action selection in corticostriatal circuits , 2011, Current Opinion in Neurobiology.

[35]  J. Nash Equilibrium Points in N-Person Games. , 1950, Proceedings of the National Academy of Sciences of the United States of America.

[36]  Daeyeol Lee,et al.  Arbitration between Action Strategies in Obsessive-Compulsive Disorder , 2016, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[37]  Yuhong Jiang,et al.  Inferior parietal lobule supports decision making under uncertainty in humans. , 2009, Cerebral cortex.

[38]  S. Haber,et al.  Reward-Related Cortical Inputs Define a Large Striatal Region in Primates That Interface with Associative Cortical Connections, Providing a Substrate for Incentive-Based Learning , 2006, The Journal of Neuroscience.

[39]  Jeffrey C. Cooper,et al.  Functional magnetic resonance imaging of reward prediction , 2005, Current opinion in neurology.

[40]  P. Glimcher,et al.  Activity in Posterior Parietal Cortex Is Correlated with the Relative Subjective Desirability of Action , 2004, Neuron.

[41]  Jonathan S. Cant,et al.  Effector-specific fields for motor preparation in the human frontal cortex , 2007, NeuroImage.

[42]  Manuela Sellitto Review for "A novel fMRI paradigm to dissociate the behavioral and neural components of mixed‐strategy decision‐making from non‐strategic decisions in humans" , 2019 .

[43]  T. Robbins,et al.  Mechanisms of cognitive set flexibility in Parkinson's disease. , 2001, Brain : a journal of neurology.

[44]  G. Waiter,et al.  Neural correlates of social exchanges during the Prisoner's Dilemma game in depression , 2016, Psychological Medicine.

[45]  John P O'Doherty,et al.  A causal account of the brain network computations underlying strategic social behavior , 2017, Nature Neuroscience.

[46]  A. Sanfey Social Decision-Making : Insights from Game Theory and Neuroscience , 2022 .

[47]  Xiao-Jing Wang Decision Making in Recurrent Neuronal Circuits , 2008, Neuron.

[48]  Minoru Asada,et al.  Different impressions of other agents obtained through social interaction uniquely modulate dorsal and ventral pathway activities in the social human brain , 2014, Cortex.

[49]  H. Seo,et al.  Lateral Intraparietal Cortex and Reinforcement Learning during a Mixed-Strategy Game , 2009, Journal of Neuroscience.

[50]  M. Brázdil,et al.  Dissecting social interaction: dual-fMRI reveals patterns of interpersonal brain-behavior relationships that dissociate among dimensions of social exchange , 2019, Social cognitive and affective neuroscience.

[51]  P. Goldman-Rakic,et al.  Common cortical and subcortical targets of the dorsolateral prefrontal and posterior parietal cortices in the rhesus monkey: evidence for a distributed neural network subserving spatially guided behavior , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[52]  W. Schultz Getting Formal with Dopamine and Reward , 2002, Neuron.

[53]  Michael J. Frank,et al.  Hold Your Horses: Impulsivity, Deep Brain Stimulation, and Medication in Parkinsonism , 2007, Science.

[54]  H. Seo,et al.  The prefrontal cortex and hybrid learning during iterative competitive games , 2011, Annals of the New York Academy of Sciences.

[55]  Patrick W. Stroman,et al.  Developmental improvements in voluntary control of behavior: Effect of preparation in the fronto-parietal network? , 2014, NeuroImage.

[56]  T. Robbins,et al.  Dopaminergic basis for deficits in working memory but not attentional set-shifting in Parkinson's disease , 2005, Neuropsychologia.

[57]  Matthew R. Kleinman,et al.  Opponent Identity Influences Value Learning in Simple Games , 2015, The Journal of Neuroscience.

[58]  John Wooders,et al.  Nash at Wimbledon: Evidence from Half a Million Serves , 2016 .

[59]  T. Schubert,et al.  The Importance of the Lateral Prefrontal Cortex for Strategic Decision Making in the Prisoner’s Dilemma , 2015, Cognitive, affective & behavioral neuroscience.

[60]  G. Glover,et al.  Dissociable Intrinsic Connectivity Networks for Salience Processing and Executive Control , 2007, The Journal of Neuroscience.

[61]  Daeyeol Lee,et al.  Impaired strategic decision making in schizophrenia , 2007, Brain Research.

[62]  Daeyeol Lee,et al.  Ubiquity and Specificity of Reinforcement Signals throughout the Human Brain , 2011, Neuron.

[63]  Akira Okada Perfect Bayesian Equilibrium and Sequential Equilibrium , 2011 .

[64]  B. Balleine,et al.  The Role of the Dorsal Striatum in Reward and Decision-Making , 2007, The Journal of Neuroscience.

[65]  M. Torrens Co-Planar Stereotaxic Atlas of the Human Brain—3-Dimensional Proportional System: An Approach to Cerebral Imaging, J. Talairach, P. Tournoux. Georg Thieme Verlag, New York (1988), 122 pp., 130 figs. DM 268 , 1990 .

[66]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[67]  Sangjoon Hahn,et al.  Response randomization of one- and two-person Rock–Paper–Scissors games in individuals with schizophrenia , 2013, Psychiatry Research.

[68]  H. Seo,et al.  Neural Basis of Strategic Decision Making , 2016, Trends in Neurosciences.

[69]  D. Barraclough,et al.  Prefrontal cortex and decision making in a mixed-strategy game , 2004, Nature Neuroscience.

[70]  A. Grace,et al.  Regulation of firing of dopaminergic neurons and control of goal-directed behaviors , 2007, Trends in Neurosciences.

[71]  Kenji Matsumoto,et al.  The Anterior Insula Tracks Behavioral Entropy during an Interpersonal Competitive Game , 2015, PloS one.

[72]  Jessica A. Grahn,et al.  The cognitive functions of the caudate nucleus , 2008, Progress in Neurobiology.

[73]  H. Seo,et al.  Temporal Filtering of Reward Signals in the Dorsal Anterior Cingulate Cortex during a Mixed-Strategy Game , 2007, The Journal of Neuroscience.

[74]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[75]  W. Newsome,et al.  Choosing the greater of two goods: neural currencies for valuation and decision making , 2005, Nature Reviews Neuroscience.

[76]  Daeyeol Lee,et al.  Distributed Coding of Actual and Hypothetical Outcomes in the Orbital and Dorsolateral Prefrontal Cortex , 2011, Neuron.

[77]  G. Pagnoni,et al.  A Neural Basis for Social Cooperation , 2002, Neuron.

[78]  Vinod Menon,et al.  Functional connectivity in the resting brain: A network analysis of the default mode hypothesis , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[79]  Steven D. Levitt,et al.  Testing Mixed-Strategy Equilibria When Players Are Heterogeneous: The Case of Penalty Kicks in Soccer , 2002 .

[80]  Ofer H. Azar,et al.  Do soccer players play the mixed-strategy Nash equilibrium? , 2011 .

[81]  Peter Bossaerts,et al.  Neural correlates of mentalizing-related computations during strategic interactions in humans , 2008, Proceedings of the National Academy of Sciences.

[82]  P. Glimcher,et al.  The Neurobiology of Decision: Consensus and Controversy , 2009, Neuron.

[83]  Peter Fransson,et al.  The precuneus/posterior cingulate cortex plays a pivotal role in the default mode network: Evidence from a partial correlation network analysis , 2008, NeuroImage.

[84]  H. Seo,et al.  Dynamic signals related to choices and outcomes in the dorsolateral prefrontal cortex. , 2007, Cerebral cortex.

[85]  Daeyeol Lee,et al.  Beyond working memory: the role of persistent activity in decision making , 2010, Trends in Cognitive Sciences.

[86]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[87]  Giorgio Coricelli,et al.  The neuroeconomics of strategic interaction , 2015, Current Opinion in Behavioral Sciences.