Dissociable but inter-related systems of cognitive control and reward during decision making: Evidence from pupillometry and event-related fMRI

Decision making involves the allocation of cognitive resources in response to expectations and feedback. Here we explored how frontal networks respond in a gambling paradigm in which uncertainty was manipulated to increase demands for cognitive control. In one experiment, pupil diameter covaried with uncertainty during decision making and with the degree to which subsequent outcomes violated reward expectations. In a second experiment, fMRI showed that both uncertainty and unexpected outcomes modulated activation in a network of frontal regions. Thus, the frontal network supports multiple phases of the decision-making process including information regarding reward uncertainty and reward outcome. In contrast, striatal activation only tracked reward delivery, suggesting a distinct reward pathway that might, under certain circumstances, oppose the frontal network. These results are consistent with the interpretation that reward signals may bias recruitment of frontal networks that are linked to allocation of cognitive resources.

[1]  R. Buckner,et al.  Functional Dissociation among Components of Remembering: Control, Perceived Oldness, and Content , 2003, The Journal of Neuroscience.

[2]  Matthew T. Kaufman,et al.  Distributed Neural Representation of Expected Value , 2005, The Journal of Neuroscience.

[3]  D. Kahneman Attention and Effort , 1973 .

[4]  Wolfgang Ellermeier,et al.  Gender differences in pain ratings and pupil reactions to painful pressure stimuli , 1995, Pain.

[5]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[6]  P S Goldman-Rakic,et al.  Dopamine D2 receptors in the cerebral cortex: distribution and pharmacological characterization with [3H]raclopride. , 1989, Proceedings of the National Academy of Sciences of the United States of America.

[7]  A. Dale,et al.  Late Onset of Anterior Prefrontal Activity during True and False Recognition: An Event-Related fMRI Study , 1997, NeuroImage.

[8]  D. Yves von Cramon,et al.  Variants of uncertainty in decision-making and their neural correlates , 2005, Brain Research Bulletin.

[9]  Mark E Wheeler,et al.  Functional-anatomic correlates of remembering and knowing , 2004, NeuroImage.

[10]  J. Hollerman,et al.  Reward processing in primate orbitofrontal cortex and basal ganglia. , 2000, Cerebral cortex.

[11]  Clay B. Holroyd,et al.  Dorsal anterior cingulate cortex shows fMRI response to internal and external error signals , 2004, Nature Neuroscience.

[12]  Jan Derrfuss,et al.  Cognitive control in the posterior frontolateral cortex: evidence from common activations in task coordination, interference control, and working memory , 2004, NeuroImage.

[13]  F. McGlone,et al.  Dopamine Transmission in the Human Striatum during Monetary Reward Tasks , 2004, The Journal of Neuroscience.

[14]  S. Petersen,et al.  Characterizing the Hemodynamic Response: Effects of Presentation Rate, Sampling Procedure, and the Possibility of Ordering Brain Activity Based on Relative Timing , 2000, NeuroImage.

[15]  J. Hollerman,et al.  Dopamine neurons report an error in the temporal prediction of reward during learning , 1998, Nature Neuroscience.

[16]  Jonathan D. Cohen,et al.  Anterior Cingulate Conflict Monitoring and Adjustments in Control , 2004, Science.

[17]  Raymond J. Dolan,et al.  Anterior cingulate activity during error and autonomic response , 2005, NeuroImage.

[18]  A. Dale,et al.  Dorsal anterior cingulate cortex: A role in reward-based decision making , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[19]  Ravi S. Menon,et al.  Intrinsic signal changes accompanying sensory stimulation: functional brain mapping with magnetic resonance imaging. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[20]  G. McCarthy,et al.  Decisions under Uncertainty: Probabilistic Context Influences Activation of Prefrontal and Parietal Cortices , 2005, The Journal of Neuroscience.

[21]  S. Petersen,et al.  Functional Anatomic Studies of Memory Retrieval for Auditory Words and Visual Pictures , 1996, The Journal of Neuroscience.

[22]  L. Nystrom,et al.  Tracking the hemodynamic responses to reward and punishment in the striatum. , 2000, Journal of neurophysiology.

[23]  J. Beatty,et al.  The pupillary system. , 2000 .

[24]  M. Botvinick,et al.  Anterior cingulate cortex, error detection, and the online monitoring of performance. , 1998, Science.

[25]  D. V. von Cramon,et al.  Error Monitoring Using External Feedback: Specific Roles of the Habenular Complex, the Reward System, and the Cingulate Motor Area Revealed by Functional Magnetic Resonance Imaging , 2003, The Journal of Neuroscience.

[26]  H. Critchley,et al.  Neural Activity in the Human Brain Relating to Uncertainty and Arousal during Anticipation , 2001, Neuron.

[27]  D. Yves von Cramon,et al.  Predicting events of varying probability: uncertainty investigated by fMRI , 2003, NeuroImage.

[28]  Jeffrey C. Cooper,et al.  Functional magnetic resonance imaging of reward prediction , 2005, Current opinion in neurology.

[29]  Jonathan D. Cohen,et al.  Computational perspectives on dopamine function in prefrontal cortex , 2002, Current Opinion in Neurobiology.

[30]  Nick F. Ramsey,et al.  Functional Anatomical Correlates of Controlled and Automatic Processing , 2001, Journal of Cognitive Neuroscience.

[31]  W. Schultz Getting Formal with Dopamine and Reward , 2002, Neuron.

[32]  Brian Knutson,et al.  Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens , 2001, The Journal of Neuroscience.

[33]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[34]  J. Maunsell Neuronal representations of cognitive state: reward or attention? , 2004, Trends in Cognitive Sciences.

[35]  D. Ariely,et al.  Beautiful Faces Have Variable Reward Value fMRI and Behavioral Evidence , 2001, Neuron.

[36]  Greg J Siegle,et al.  Use of concurrent pupil dilation assessment to inform interpretation and analysis of fMRI data , 2003, NeuroImage.

[37]  Samuel M. McClure,et al.  Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum , 2003, Neuron.

[38]  K. Berman,et al.  Cerebral Cortex doi:10.1093/cercor/bhj004 Neural Coding of Distinct Statistical Properties of Reward Information in Humans , 2005 .

[39]  Brian Knutson,et al.  Dissociation of reward anticipation and outcome with event-related fMRI , 2001, Neuroreport.

[40]  D. Kahneman,et al.  Functional Imaging of Neural Responses to Expectancy and Experience of Monetary Gains and Losses tasks with monetary payoffs , 2001 .

[41]  T. Robbins,et al.  Choosing between Small, Likely Rewards and Large, Unlikely Rewards Activates Inferior and Orbital Prefrontal Cortex , 1999, The Journal of Neuroscience.

[42]  J. Duncan,et al.  Common regions of the human frontal lobe recruited by diverse cognitive demands , 2000, Trends in Neurosciences.

[43]  M. Brass,et al.  The role of the inferior frontal junction area in cognitive control , 2005, Trends in Cognitive Sciences.

[44]  S. N. Haber,et al.  The organization of midbrain projections to the ventral striatum in the primate , 1994, Neuroscience.

[45]  J. Beatty Task-evoked pupillary responses, processing load, and the structure of processing resources. , 1982, Psychological bulletin.

[46]  K. R. Ridderinkhof,et al.  The Role of the Medial Frontal Cortex in Cognitive Control , 2004, Science.

[47]  Gregory G. Brown,et al.  Error Rate and Outcome Predictability Affect Neural Activation in Prefrontal Cortex and Anterior Cingulate during Decision-Making , 2002, NeuroImage.

[48]  Clay B. Holroyd,et al.  Errors in reward prediction are re£ected in the event-related brain potential , 2003 .

[49]  J. O'Doherty,et al.  Neural Responses during Anticipation of a Primary Taste Reward , 2002, Neuron.

[50]  Karl J. Friston,et al.  Dissociable Neural Responses in Human Reward Systems , 2000, The Journal of Neuroscience.

[51]  M. Brass,et al.  Involvement of the inferior frontal junction in cognitive control: Meta‐analyses of switching and Stroop studies , 2005, Human brain mapping.

[52]  Jonathan D. Cohen,et al.  An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. , 2005, Annual review of neuroscience.

[53]  R. Buckner,et al.  Common Prefrontal Regions Coactivate with Dissociable Posterior Regions during Controlled Semantic and Phonological Tasks , 2002, Neuron.

[54]  Joshua W. Brown,et al.  Learned Predictions of Error Likelihood in the Anterior Cingulate Cortex , 2005, Science.

[55]  Brian Knutson,et al.  FMRI Visualization of Brain Activity during a Monetary Incentive Delay Task , 2000, NeuroImage.

[56]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[57]  Karl J. Friston,et al.  Analysis of fMRI Time-Series Revisited , 1995, NeuroImage.

[58]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[59]  A. Dale,et al.  Functional–Anatomic Study of Episodic Retrieval II. Selective Averaging of Event-Related fMRI Trials to Test the Retrieval Success Hypothesis , 1998, NeuroImage.

[60]  Brian Knutson,et al.  Amphetamine Modulates Human Incentive Processing , 2004, Neuron.

[61]  Jonathan D. Cohen,et al.  Computational roles for dopamine in behavioural control , 2004, Nature.

[62]  A. Dale,et al.  Functional-Anatomic Correlates of Object Priming in Humans Revealed by Rapid Presentation Event-Related fMRI , 1998, Neuron.

[63]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[64]  R. Turner,et al.  Dynamic magnetic resonance imaging of human brain activity during primary sensory stimulation. , 1992, Proceedings of the National Academy of Sciences of the United States of America.

[65]  M. Botvinick,et al.  Conflict monitoring and cognitive control. , 2001, Psychological review.

[67]  S. Hyman,et al.  Acute Effects of Cocaine on Human Brain Activity and Emotion , 1997, Neuron.

[68]  Randy L Buckner,et al.  Functional–Anatomic Correlates of Control Processes in Memory , 2003, The Journal of Neuroscience.

[69]  D. Brooks,et al.  Evidence for striatal dopamine release during a video game , 1998, Nature.

[70]  Randy L. Buckner,et al.  Rapid self-paced event-related functional MRI , 2001, NeuroImage.

[71]  D. Heeger,et al.  Linear Systems Analysis of Functional Magnetic Resonance Imaging in Human V1 , 1996, The Journal of Neuroscience.

[72]  Karl J. Friston,et al.  Temporal Difference Models and Reward-Related Learning in the Human Brain , 2003, Neuron.

[73]  P. Montague,et al.  Activity in human ventral striatum locked to errors of reward prediction , 2002, Nature Neuroscience.

[74]  Matthew Flatt,et al.  PsyScope: An interactive graphic system for designing and controlling experiments in the psychology laboratory using Macintosh computers , 1993 .

[75]  T. Paus,et al.  Regional differences in the effects of task difficulty and motor output on blood flow response in the human anterior cingulate cortex: a review of 107 PET activation studies , 1998, Neuroreport.

[76]  Karl J. Friston,et al.  Analysis of fMRI Time-Series Revisited—Again , 1995, NeuroImage.

[77]  Samuel M. McClure,et al.  Predictability Modulates Human Brain Response to Reward , 2001, The Journal of Neuroscience.

[78]  Tor D. Wager,et al.  Common and unique components of response inhibition revealed by fMRI , 2005, NeuroImage.

[79]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[80]  Benjamin J. Shannon,et al.  Functional-Anatomic Correlates of Memory Retrieval That Suggest Nontraditional Processing Roles for Multiple Distinct Regions within Posterior Parietal Cortex , 2004, The Journal of Neuroscience.

[81]  Samuel M. McClure,et al.  The Neural Substrates of Reward Processing in Humans: The Modern Role of fMRI , 2004, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[82]  A. Dale,et al.  Selective averaging of rapidly presented individual trials using fMRI , 1997, Human brain mapping.

[83]  Randy L. Buckner,et al.  Functional–Anatomic Correlates of Sustained and Transient Processing Components Engaged during Controlled Retrieval , 2003, The Journal of Neuroscience.

[84]  K. Doya,et al.  A Neural Correlate of Reward-Based Behavioral Learning in Caudate Nucleus: A Functional Magnetic Resonance Imaging Study of a Stochastic Decision Task , 2004, The Journal of Neuroscience.

[85]  M. Delgado,et al.  Dorsal striatum responses to reward and punishment: Effects of valence and magnitude manipulations , 2003, Cognitive, affective & behavioral neuroscience.