Outcome contingency selectively affects the neural coding of outcomes but not of tasks

Humans make choices every day, which are often intended to lead to desirable outcomes. While we often have some degree of control over the outcomes of our actions, in many cases this control remains limited. Here, we investigate the effect of control over outcomes on the neural correlates of outcome valuation and implementation of behavior, as desired outcomes can only be reached if choices are implemented as intended. In a value-based decision-making task, reward outcomes were either contingent on trial-by-trial choices between two different tasks, or were unrelated to these choices. Using fMRI, multivariate pattern analysis, and model-based neuroscience methods, we identified reward representations in a large network including the striatum, dorso-medial prefrontal cortex (dmPFC) and parietal cortex. These representations were amplified when rewards were contingent on subjects9 choices. We further assessed the implementation of chosen tasks by identifying brain regions encoding tasks during a preparation or maintenance phase, and found them to be encoded in the dmPFC and parietal cortex. Importantly, outcome contingency did not affect neural coding of chosen tasks. This suggests that controlling choice outcomes selectively affects the neural coding of these outcomes, but has no effect on the means to reach them. Overall, our findings highlight the role of the dmPFC and parietal cortex in processing of value-related and task-related information, linking motivational and control-related processes in the brain. These findings inform current debates on the neural basis of motivational and cognitive control, as well as their interaction.

[1]  B. Libet,et al.  Time of conscious intention to act in relation to onset of cerebral activity (readiness-potential). The unconscious initiation of a freely voluntary act. , 1983, Brain : a journal of neurology.

[2]  C. F. Kao,et al.  The efficient assessment of need for cognition. , 1984, Journal of personality assessment.

[3]  Richard S. Sutton,et al.  Time-Derivative Models of Pavlovian Reinforcement , 1990 .

[4]  M. Gabriel,et al.  Learning and Computational Neuroscience: Foundations of Adaptive Networks , 1990 .

[5]  C. Carver,et al.  Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS Scales , 1994 .

[6]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[7]  J. Patton,et al.  Factor structure of the Barratt impulsiveness scale. , 1995, Journal of clinical psychology.

[8]  D. Meyer,et al.  Executive control of cognitive processes in task switching. , 2001, Journal of experimental psychology. Human perception and performance.

[9]  C. Ávila,et al.  The Sensitivity to Punishment and Sensitivity to Reward Questionnaire (SPSRQ) as a measure of Gray's anxiety and impulsivity dimensions. , 2001 .

[10]  S. Kapur,et al.  Direct Activation of the Ventral Striatum in Anticipation of Aversive Stimuli , 2003, Neuron.

[11]  Brian Knutson,et al.  A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event-related fMRI , 2003, NeuroImage.

[12]  E. Miller,et al.  Neural circuits subserving the retrieval and maintenance of abstract rules. , 2003, Journal of neurophysiology.

[13]  David D. Cox,et al.  Functional magnetic resonance imaging (fMRI) “brain reading”: detecting and classifying distributed patterns of fMRI activity in human visual cortex , 2003, NeuroImage.

[14]  Okihide Hikosaka,et al.  Reward-Dependent Gain and Bias of Visual Responses in Primate Superior Colliculus , 2003, Neuron.

[15]  D. Yves von Cramon,et al.  Predicting events of varying probability: uncertainty investigated by fMRI , 2003, NeuroImage.

[16]  W. Newsome,et al.  Matching Behavior and the Representation of Value in the Parietal Cortex , 2004, Science.

[17]  E. Murray,et al.  Bilateral Orbital Prefrontal Cortex Lesions in Rhesus Monkeys Disrupt Choices Guided by Both Reward Value and Reward Contingency , 2004, The Journal of Neuroscience.

[18]  Rebecca Elliott,et al.  Instrumental responding for rewards is associated with enhanced neuronal response in subcortical reward systems , 2004, NeuroImage.

[19]  David R. Anderson,et al.  Multimodel Inference , 2004 .

[20]  Tom M. Mitchell,et al.  Learning to Decode Cognitive States from Brain Images , 2004, Machine Learning.

[21]  M. Delgado,et al.  Modulation of Caudate Activity by Action Contingency , 2004, Neuron.

[22]  Catherine M Arrington,et al.  PSYCHOLOGICAL SCIENCE Research Article The Cost of a Voluntary Task Switch , 2022 .

[23]  F. Tong,et al.  Decoding the visual and subjective contents of the human brain , 2005, Nature Neuroscience.

[24]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[25]  J. C. Johnston,et al.  On the limits of advance preparation for a task switch: do people prepare all the task some of the time or some of the task all the time? , 2005, Journal of experimental psychology. Human perception and performance.

[26]  S. Inati,et al.  An fMRI study of reward-related probability learning , 2005, NeuroImage.

[27]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[28]  Rainer Goebel,et al.  Information-based functional brain mapping. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[29]  K. Doya,et al.  The computational neurobiology of learning and reward , 2006, Current Opinion in Neurobiology.

[30]  J. O'Doherty,et al.  Model‐Based fMRI and Its Application to Reward Learning and Decision Making , 2007, Annals of the New York Academy of Sciences.

[31]  E. Wagenmakers A practical solution to the pervasive problems ofp values , 2007, Psychonomic bulletin & review.

[32]  Timothy E. J. Behrens,et al.  Adaptive decision making and value in the anterior cingulate cortex , 2007, NeuroImage.

[33]  M. McCloskey,et al.  IQ and nonplanning impulsivity are independently associated with delay discounting in middle-aged adults , 2007 .

[34]  R. Passingham,et al.  Reading Hidden Intentions in the Human Brain , 2007, Current Biology.

[35]  Keiji Tanaka,et al.  Medial prefrontal cell activity signaling prediction errors of action values , 2007, Nature Neuroscience.

[36]  Jonathan W. Peirce,et al.  PsychoPy—Psychophysics software in Python , 2007, Journal of Neuroscience Methods.

[37]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[38]  J. O'Doherty,et al.  Decoding the neural substrates of reward-related decision making with functional MRI , 2007, Proceedings of the National Academy of Sciences.

[39]  M. Brass,et al.  Unconscious determinants of free decisions in the human brain , 2008, Nature Neuroscience.

[40]  W. Senn,et al.  Dopamine increases the gain of the input–output response of rat prefrontal pyramidal neurons. J. Neurophysiol. (in press). doi: 10.1152/jn.01098.2007 [epub ahead of print , 2008 .

[41]  K. Doya Modulators of decision making , 2008, Nature Neuroscience.

[42]  John Duncan,et al.  Hierarchical coding for sequential task events in the monkey prefrontal cortex , 2008, Proceedings of the National Academy of Sciences.

[43]  D. Eckstein,et al.  Rule-Selection and Action-Selection have a Shared Neuroanatomical Basis in the Human Prefrontal and Parietal Cortex , 2008, Cerebral cortex.

[44]  Stefan Bode,et al.  Decoding sequential stages of task preparation in the human brain , 2009, NeuroImage.

[45]  T. Braver,et al.  Attention, intention, and strategy in preparatory control , 2009, Neuropsychologia.

[46]  Jeffrey N. Rouder,et al.  Bayesian t tests for accepting and rejecting the null hypothesis , 2009, Psychonomic bulletin & review.

[47]  J. Duncan The multiple-demand (MD) system of the primate brain: mental programs for intelligent behaviour , 2010, Trends in Cognitive Sciences.

[48]  Hannah S. Locke,et al.  Prefrontal cortex mediation of cognitive enhancement in rewarding motivational contexts , 2010, Proceedings of the National Academy of Sciences.

[49]  P. Glimcher,et al.  Testing the Reward Prediction Error Hypothesis with an Axiomatic Model , 2010, The Journal of Neuroscience.

[50]  P. Fossati,et al.  Different brain structures related to self- and external-agency attribution: a brief review and meta-analysis , 2011, Brain Structure and Function.

[51]  Christian F. Doeller,et al.  Anterior Hippocampus and Goal-Directed Spatial Decision Making , 2011, The Journal of Neuroscience.

[52]  J. Duncan,et al.  Adaptive Coding of Task-Relevant Information in Human Frontoparietal Cortex , 2011, The Journal of Neuroscience.

[53]  S. Gilbert Decoding the Content of Delayed Intentions , 2011, The Journal of Neuroscience.

[54]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[55]  Dorit Wenke,et al.  The shielding function of task sets and its relaxation during task switching. , 2011, Journal of experimental psychology. Learning, memory, and cognition.

[56]  G. Dreisbach,et al.  The role of affect and reward in the conflict-triggered adjustment of cognitive control , 2012, Front. Hum. Neurosci..

[57]  James V. Haxby,et al.  Multivariate pattern analysis of fMRI: The early beginnings , 2012, NeuroImage.

[58]  Johan D. Carlin,et al.  Choosing the Rules: Distinct and Overlapping Frontoparietal Representations of Task Rules for Perceptual Decisions , 2013, The Journal of Neuroscience.

[59]  E. Murray,et al.  The role of the anterior cingulate cortex in choices based on reward value and reward contingency. , 2013, Cerebral cortex.

[60]  Jonathan D. Cohen,et al.  Confounds in multivariate pattern analysis: Theory and rule representation case study , 2013, NeuroImage.

[61]  N. Sigala,et al.  Dynamic Coding for Cognitive Control in Prefrontal Cortex , 2013, Neuron.

[62]  J. Haynes,et al.  Predicting free choices for abstract intentions , 2013, Proceedings of the National Academy of Sciences.

[63]  Franziska M. Korb,et al.  Affective Modulation of Cognitive Control is Determined by Performance-Contingency and Mediated by Ventromedial Prefrontal and Cingulate Cortex , 2013, The Journal of Neuroscience.

[64]  Nancy Kanwisher,et al.  Broad domain generality in focal regions of frontal and parietal cortex , 2013, Proceedings of the National Academy of Sciences.

[65]  D. Hassabis,et al.  Foraging under Competition: The Neural Basis of Input-Matching in Humans , 2013, The Journal of Neuroscience.

[66]  Jonathan D. Cohen,et al.  The Computational and Neural Basis of Cognitive Control: Charted Territory and New Frontiers , 2014, Cogn. Sci..

[67]  Yu Bai,et al.  Dual learning processes underlying human decision-making in reversal learning tasks: functional significance and evidence from the model fit to human behavior , 2014, Front. Psychol..

[68]  W. Fias,et al.  Dissociating contributions of ACC and vmPFC in reward prediction, outcome, and choice , 2014, Neuropsychologia.

[69]  Michael L. Waskom,et al.  Frontoparietal Representations of Task Context Support the Flexible Control of Goal-Directed Cognition , 2014, The Journal of Neuroscience.

[70]  John-Dylan Haynes,et al.  Disentangling neural representations of value and salience in the human brain , 2014, Proceedings of the National Academy of Sciences.

[71]  Polina Golland,et al.  Coping with confounds in multivoxel pattern analysis: What should we do about reaction time differences? A comment on Todd, Nystrom & Cohen 2013 , 2014, NeuroImage.

[72]  Kingson Man,et al.  Multivariate cross-classification: applying machine learning techniques to characterize abstraction in neural representations , 2015, Front. Hum. Neurosci..

[73]  J. Haynes A Primer on Pattern-Based Approaches to fMRI: Principles, Pitfalls, and Perspectives , 2015, Neuron.

[74]  Anina N. Rich,et al.  Flexible Coding of Task Rules in Frontoparietal Cortex: An Adaptive System for Flexible Cognitive Control , 2015, Journal of Cognitive Neuroscience.

[75]  Jörg Rieskamp,et al.  An Introduction to Bayesian Hypothesis Testing for Management Research , 2015 .

[76]  I. Momennejad,et al.  The Role of the Parietal Cortex in the Representation of Task–Reward Associations , 2015, The Journal of Neuroscience.

[77]  E. Wagenmakers,et al.  An Introduction to Model-Based Cognitive Neuroscience , 2015, Springer New York.

[78]  John-Dylan Haynes,et al.  The Neural Representation of Voluntary Task-Set Selection in Dynamic Environments. , 2015, Cerebral cortex.

[79]  K. Cheng,et al.  Neural basis of decision making guided by emotional outcomes. , 2015, Journal of neurophysiology.

[80]  Martin N. Hebart,et al.  The Decoding Toolbox (TDT): a versatile software package for multivariate analyses of functional imaging data , 2015, Front. Neuroinform..

[81]  Jan De Houwer,et al.  Potentiation of the startle reflex is in line with contingency reversal instructions rather than the conditioning history , 2016, Biological Psychology.

[82]  Michael W. Cole,et al.  Reward Motivation Enhances Task Coding in Frontoparietal Cortex. , 2016, Cerebral cortex.

[83]  Wolfram Schultz,et al.  Dopamine reward prediction-error signalling: a two-component response , 2016, Nature Reviews Neuroscience.

[84]  Hans Knutsson,et al.  Cluster failure: Why fMRI inferences for spatial extent have inflated false-positive rates , 2016, Proceedings of the National Academy of Sciences.

[85]  John-Dylan Haynes,et al.  Similar coding of freely chosen and externally cued intentions in a fronto-parietal network , 2016, NeuroImage.

[86]  Lasse S Loose,et al.  Switch independent task representations in frontal and parietal cortex , 2017, bioRxiv.

[87]  David Badre,et al.  Just above chance: is it harder to decode information from human prefrontal cortex BOLD signals? , 2017, bioRxiv.

[88]  Tobias Egner,et al.  Dynamic Trial-by-Trial Recoding of Task-Set Representations in the Frontoparietal Cortex Mediates Behavioral Flexibility , 2017, The Journal of Neuroscience.

[89]  David Badre,et al.  Working Memory Load Strengthens Reward Prediction Errors , 2017, The Journal of Neuroscience.

[90]  Jens Schwarzbach,et al.  Decoding of auditory and tactile perceptual decisions in parietal cortex , 2017, NeuroImage.

[91]  David Badre,et al.  Just above Chance: Is It Harder to Decode Information from Prefrontal Cortex Hemodynamic Activity Patterns? , 2018, Journal of Cognitive Neuroscience.

[92]  Etienne Koechlin,et al.  The Neuro-Computational Architecture of Value-Based Selection in the Human Brain , 2017, Cerebral cortex.

[93]  David Wisniewski Context-Dependence and Context-Invariance in the Neural Coding of Intentional Action , 2018, Front. Psychol..

[94]  A. R. Otto,et al.  Cognitive capacity limitations and Need for Cognition differentially predict reward-induced cognitive effort expenditure , 2018, Cognition.