Independent circuits in basal ganglia and cortex for the processing of reward and precision feedback

&NA; In order to understand human decision making it is necessary to understand how the brain uses feedback to guide goal‐directed behavior. The ventral striatum (VS) appears to be a key structure in this function, responding strongly to explicit reward feedback. However, recent results have also shown striatal activity following correct task performance even in the absence of feedback. This raises the possibility that, in addition to processing external feedback, the dopamine‐centered “reward circuit” might regulate endogenous reinforcement signals, like those triggered by satisfaction in accurate task performance. Here we use functional magnetic resonance imaging (fMRI) to test this idea. Participants completed a simple task that garnered both reward feedback and feedback about the precision of performance. Importantly, the design was such that we could manipulate information about the precision of performance within different levels of reward magnitude. Using parametric modulation and functional connectivity analysis we identified brain regions sensitive to each of these signals. Our results show a double dissociation: frontal and posterior cingulate regions responded to explicit reward but were insensitive to task precision, whereas the dorsal striatum ‐ and putamen in particular ‐ was insensitive to reward but responded strongly to precision feedback in reward‐present trials. Both types of feedback activated the VS, and sensitivity in this structure to precision feedback was predicted by personality traits related to approach behavior and reward responsiveness. Our findings shed new light on the role of specific brain regions in integrating different sources of feedback to guide goal‐directed behavior. HighlightsWe investigated the sensitivity of the reward system to external reward and task‐precision feedback.Frontal and posterior cingulate regions responded to explicit reward but were insensitive to task precision.The posterior putamen was insensitive to reward but responded strongly to precision feedback in reward‐present trials.Both external reward and precision feedback activated the ventral striatum.The sensitivity of the ventral striatum to precision feedback was predicted by reward‐related personality traits.

[1]  Elizabeth Tricomi,et al.  Modulation of ventral striatal activity by cognitive effort , 2017, NeuroImage.

[2]  N. Bunzeck,et al.  Absolute Coding of Stimulus Novelty in the Human Substantia Nigra/VTA , 2006, Neuron.

[3]  D. V. von Cramon,et al.  Error Monitoring Using External Feedback: Specific Roles of the Habenular Complex, the Reward System, and the Cingulate Motor Area Revealed by Functional Magnetic Resonance Imaging , 2003, The Journal of Neuroscience.

[4]  Jeffrey C. Cooper,et al.  Functional magnetic resonance imaging of reward prediction , 2005, Current opinion in neurology.

[5]  Mark A. Elliott,et al.  Striatal intrinsic reinforcement signals during recognition memory: relationship to response bias and dysregulation in schizophrenia , 2011, Front. Behav. Neurosci..

[6]  Russell A. Poldrack,et al.  Orthogonalization of Regressors in fMRI Models , 2015, PloS one.

[7]  Jesper Andersson,et al.  Valid conjunction inference with the minimum statistic , 2005, NeuroImage.

[8]  W. Schultz.  Behavioral theories and the neurophysiology of reward. , 2006, Annual review of psychology.

[9]  M. Petrides,et al.  Functional role of the basal ganglia in the planning and execution of actions , 2006, Annals of neurology.

[10]  J. O'Doherty,et al.  Neural Responses during Anticipation of a Primary Taste Reward , 2002, Neuron.

[11]  C. Carver,et al.  Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS Scales , 1994 .

[12]  Carol A. Seger,et al.  Dissociation between Striatal Regions while Learning to Categorize via Feedback and via Observation , 2007, Journal of Cognitive Neuroscience.

[13]  L. Shah,et al.  Functional magnetic resonance imaging. , 2010, Seminars in roentgenology.

[14]  A. Bonci,et al.  Role of Dopamine Neurons in Reward and Aversion: A Synaptic Plasticity Perspective , 2015, Neuron.

[15]  Brian Knutson,et al.  A region of mesial prefrontal cortex tracks monetarily rewarding outcomes: characterization with rapid event-related fMRI , 2003, NeuroImage.

[16]  Ruth M. Krebs,et al.  Novelty increases the mesolimbic functional connectivity of the substantia nigra/ventral tegmental area (SN/VTA) during reward anticipation: Evidence from high-resolution fMRI , 2011, NeuroImage.

[17]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[18]  P. Fletcher,et al.  Faculty Opinions recommendation of A selective role for dopamine in stimulus-reward learning. , 2011 .

[19]  Jane R. Garrison,et al.  Prediction error in reinforcement learning: A meta-analysis of neuroimaging studies , 2013, Neuroscience & Biobehavioral Reviews.

[20]  Mark A. Elliott,et al.  Being right is its own reward: Load and performance related ventral striatum activation to correct responses during a working memory task in youth , 2012, NeuroImage.

[21]  Benjamin Y. Hayden,et al.  Posterior Cingulate Cortex Mediates Outcome-Contingent Allocation of Behavior , 2008, Neuron.

[22]  Scott A. Huettel,et al.  Functional Significance of Striatal Responses during Episodic Decisions: Recovery or Goal Attainment? , 2010, The Journal of Neuroscience.

[23]  Shawn W. Ell.  Contributions of the putamen to cognitive function , 2011 .

[24]  Wolfram Schultz,et al.  Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. , 2003, Journal of neurophysiology.

[25]  S. Floresco.  The nucleus accumbens: an interface between cognition, emotion, and action. , 2015, Annual review of psychology.

[26]  S. Pollmann,et al.  Putamen Activation Represents an Intrinsic Positive Prediction Error Signal for Visual Search in Repeated Configurations , 2016, The open neuroimaging journal.

[27]  W. Schultz,et al.  Responses to reward in monkey dorsal and ventral striatum , 2004, Experimental Brain Research.

[28]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[29]  J. O'Doherty,et al.  Lights, Camembert, Action! The Role of Human Orbitofrontal Cortex in Encoding Stimuli, Rewards, and Choices , 2007, Annals of the New York Academy of Sciences.

[30]  P. Sterzer,et al.  Mesolimbic confidence signals guide perceptual learning in the absence of external feedback , 2016, eLife.

[31]  J. C. Crowley,et al.  Saccade Reward Signals in Posterior Cingulate Cortex , 2003, Neuron.

[32]  Jin Fan,et al.  Common and distinct networks underlying reward valence and processing stages: A meta-analysis of functional neuroimaging studies , 2011, Neuroscience & Biobehavioral Reviews.

[33]  M. Peelen,et al.  Neural Mechanisms of Incentive Salience in Naturalistic Human Vision , 2015, Neuron.

[34]  Jean-Baptiste Poline,et al.  Distinct striatal regions support movement selection, preparation and execution , 2004, Neuroreport.

[35]  K. Berridge,et al.  What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? , 1998, Brain Research Reviews.

[36]  M. Delgado,et al.  Modulation of Caudate Activity by Action Contingency , 2004, Neuron.

[37]  Stefan Pollmann,et al.  Striatal activations signal prediction errors on confidence in the absence of external feedback , 2012, NeuroImage.

[38]  W. Schultz.  Updating dopamine reward signals , 2013, Current Opinion in Neurobiology.

[39]  Karl J. Friston,et al.  Statistical parametric maps in functional imaging: A general linear approach , 1994 .

[40]  R. Poldrack,et al.  Ventral–striatal/nucleus–accumbens sensitivity to prediction errors during classification learning , 2006, Human brain mapping.

[41]  RuppinEytan,et al.  Actor-critic models of the basal ganglia , 2002 .

[42]  Sterling C. Johnson,et al.  A generalized form of context-dependent psychophysiological interactions (gPPI): A comparison to standard approaches , 2012, NeuroImage.

[43]  C. Padoa-Schioppa,et al.  Neurons in the orbitofrontal cortex encode economic value , 2006, Nature.

[44]  Nicolas W. Schuck,et al.  Reduced Striatal Responses to Reward Prediction Errors in Older Compared with Younger Adults , 2013, The Journal of Neuroscience.

[45]  Karl J. Friston,et al.  Modeling regional and psychophysiologic interactions in fMRI: the importance of hemodynamic deconvolution , 2003, NeuroImage.

[46]  John M. Pearson,et al.  Posterior cingulate cortex: adapting behavior to a changing world , 2011, Trends in Cognitive Sciences.

[47]  W. Schultz.  Multiple reward signals in the brain , 2000, Nature Reviews Neuroscience.

[48]  W. Schultz,et al.  Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. , 2000, Journal of neurophysiology.

[49]  S. Wise,et al.  Frontal pole cortex: encoding ends at the end of the endbrain , 2011, Trends in Cognitive Sciences.

[50]  K. Fliessbach,et al.  Dissociation of BOLD responses to reward prediction errors and reward receipt by a model comparison , 2012, The European journal of neuroscience.

[51]  M. Turatto,et al.  Monetary Reward Modulates Task-Irrelevant Perceptual Learning for Invisible Stimuli , 2015, PloS one.

[52]  J. Horvitz.  Stimulus–response and response–outcome learning mechanisms in the striatum , 2009, Behavioural Brain Research.

[53]  J. O'Doherty,et al.  Model‐Based fMRI and Its Application to Reward Learning and Decision Making , 2007, Annals of the New York Academy of Sciences.

[54]  Eytan Ruppin,et al.  Actor-critic models of the basal ganglia: new anatomical and computational perspectives , 2002, Neural Networks.

[55]  M Zaitsev,et al.  Point spread function mapping with parallel imaging techniques and high acceleration factors: Fast, robust, and flexible method for echo‐planar imaging distortion correction , 2004, Magnetic resonance in medicine.

[56]  B. Hayden,et al.  Electrophysiological correlates of default-mode processing in macaque posterior cingulate cortex , 2009, Proceedings of the National Academy of Sciences.

[57]  Kenji Matsumoto,et al.  Neural basis of the undermining effect of monetary reward on intrinsic motivation , 2010, Proceedings of the National Academy of Sciences.

[58]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[59]  M. Brammer,et al.  Progressive increase of frontostriatal brain activation from childhood to adulthood during event‐related tasks of cognitive control , 2006, Human brain mapping.

[60]  Samuel M. McClure,et al.  The Neural Substrates of Reward Processing in Humans: The Modern Role of fMRI , 2004, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[61]  Samuel M. McClure,et al.  Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum , 2003, Neuron.

[62]  J. Jankowski,et al.  Distinct striatal regions for planning and executing novel and automated movement sequences , 2009, NeuroImage.

[63]  John M. Pearson,et al.  Neurons in Posterior Cingulate Cortex Signal Exploratory Decisions in a Dynamic Multioption Choice Task , 2009, Current Biology.

[64]  J. O'Doherty,et al.  Is Avoiding an Aversive Outcome Rewarding? Neural Substrates of Avoidance Learning in the Human Brain , 2006, PLoS biology.

[65]  David Pascucci,et al.  Immediate Effect of Internal Reward on Visual Adaptation , 2013, Psychological science.

[66]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[67]  James L. McClelland,et al.  Performance Feedback Drives Caudate Activation in a Phonological Learning Task , 2006, Journal of Cognitive Neuroscience.

[68]  M. Delgado,et al.  Motivation-dependent responses in the human caudate nucleus. , 2004, Cerebral cortex.

[69]  Stefan Pollmann,et al.  A universal role of the ventral striatum in reward-based learning: Evidence from human studies , 2014, Neurobiology of Learning and Memory.

[70]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[71]  D. Sharp,et al.  The role of the posterior cingulate cortex in cognition and disease. , 2014, Brain : a journal of neurology.

[72]  Lucia Mannetti,et al.  Validità della versione italiana delle scale BIS/BAS di Carver e White (1994): generalizzabilità della struttura e relazioni con costrutti affini , 2002 .

[73]  Elizabeth Tricomi,et al.  Feedback signals in the caudate reflect goal achievement on a declarative memory task , 2008, NeuroImage.

[74]  Nikolaus R. McFarland,et al.  Striatonigrostriatal Pathways in Primates Form an Ascending Spiral from the Shell to the Dorsolateral Striatum , 2000, The Journal of Neuroscience.

[75]  D G Pelli,et al.  The VideoToolbox software for visual psychophysics: transforming numbers into movies. , 1997, Spatial vision.

[76]  B. Balleine,et al.  A specific role for posterior dorsolateral striatum in human habit learning , 2009, The European journal of neuroscience.

[77]  J. O'Doherty,et al.  Reward representations and reward-related learning in the human brain: insights from neuroimaging , 2004, Current Opinion in Neurobiology.

[78]  S. Pollmann,et al.  Comparing the Neural Basis of Monetary Reward and Cognitive Feedback during Information-Integration Category Learning , 2010, The Journal of Neuroscience.

[79]  Brian Knutson,et al.  Anticipation of Increasing Monetary Reward Selectively Recruits Nucleus Accumbens , 2001, The Journal of Neuroscience.