Neural Correlates of Instrumental Contingency Learning: Differential Effects of Action–Reward Conjunction and Disjunction

Contingency theories of goal-directed action propose that experienced disjunctions between an action and its specific consequences, as well as conjunctions between these events, contribute to encoding the action–outcome association. Although considerable behavioral research in rats and humans has provided evidence for this proposal, relatively little is known about the neural processes that contribute to the two components of the contingency calculation. Specifically, while recent findings suggest that the influence of action–outcome conjunctions on goal-directed learning is mediated by a circuit involving ventromedial prefrontal, medial orbitofrontal cortex, and dorsomedial striatum, the neural processes that mediate the influence of experienced disjunctions between these events are unknown. Here we show differential responses to probabilities of conjunctive and disjunctive reward deliveries in the ventromedial prefrontal cortex, the dorsomedial striatum, and the inferior frontal gyrus. Importantly, activity in the inferior parietal lobule and the left middle frontal gyrus varied with a formal integration of the two reward probabilities, ΔP, as did response rates and explicit judgments of the causal efficacy of the action.

[1]  K. Doya,et al.  Representation of Action-Specific Reward Values in the Striatum , 2005, Science.

[2]  B. Balleine,et al.  Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual Action , 2010, Neuropsychopharmacology.

[3]  A. Dickinson,et al.  Instrumental judgment and performance under variations in action-outcome contingency and contiguity , 1991, Memory & cognition.

[4]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[5]  J. O'Doherty,et al.  Neural Responses during Anticipation of a Primary Taste Reward , 2002, Neuron.

[6]  Paul J. Laurienti,et al.  An automated method for neuroanatomic and cytoarchitectonic atlas-based interrogation of fMRI data sets , 2003, NeuroImage.

[7]  Vivian V. Valentin,et al.  Determining the Neural Substrates of Goal-Directed Learning in the Human Brain , 2007, The Journal of Neuroscience.

[8]  B. Skinner What is the experimental analysis of behavior? , 1966, Journal of the experimental analysis of behavior.

[9]  A. Dale,et al.  Dorsal anterior cingulate cortex: A role in reward-based decision making , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[10]  B. Hayden,et al.  Distinct Value Signals in Anterior and Posterior Ventromedial Prefrontal Cortex , 2010, The Journal of Neuroscience.

[11]  Jonathan D. Cohen,et al.  Improved Assessment of Significant Activation in Functional Magnetic Resonance Imaging (fMRI): Use of a Cluster‐Size Threshold , 1995, Magnetic resonance in medicine.

[12]  Edward A. Wasserman,et al.  Response-outcome contingency: Behavioral and judgmental effects of appetitive and aversive outcomes with college students , 1985 .

[13]  J. Gläscher,et al.  Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. , 2009, Cerebral cortex.

[14]  B. Balleine,et al.  Calculating Consequences: Brain Systems That Encode the Causal Effects of Actions , 2008, The Journal of Neuroscience.

[15]  B. Balleine,et al.  The role of prelimbic cortex in instrumental conditioning , 2003, Behavioural Brain Research.

[16]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[17]  S. Inati,et al.  An fMRI study of reward-related probability learning , 2005, NeuroImage.

[18]  B. Balleine,et al.  The role of the dorsomedial striatum in instrumental conditioning , 2005, The European journal of neuroscience.

[19]  D. Ortu,et al.  Experience-dependent changes in human brain activation during contingency learning , 2010, Neuroscience.

[20]  H. Seo,et al.  Lateral Intraparietal Cortex and Reinforcement Learning during a Mixed-Strategy Game , 2009, Journal of Neuroscience.

[21]  Jean-Luc Anton,et al.  Region of interest analysis using an SPM toolbox , 2010 .

[22]  M. Schlund,et al.  Integrating functional neuroimaging and human operant research: brain activation correlated with presentation of discriminative stimuli. , 2005, Journal of the experimental analysis of behavior.

[23]  L. J. Hammond The effect of contingency upon the appetitive conditioning of free-operant behavior. , 1980, Journal of the experimental analysis of behavior.

[24]  M. Delgado,et al.  Modulation of Caudate Activity by Action Contingency , 2004, Neuron.

[25]  Heinrich Sauer,et al.  The neural correlates of reward-related trial-and-error learning: an fMRI study with a probabilistic learning task. , 2008, Learning & memory.

[26]  Benjamin J. Tamber-Rosenau,et al.  Avoiding non-independence in fMRI data analysis: Leave one subject out , 2010, NeuroImage.

[27]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[28]  Antonio Rangel,et al.  Economic choices can be made using only stimulus values , 2010, Proceedings of the National Academy of Sciences.

[29]  J. O'Doherty,et al.  Dissociating Valence of Outcome from Behavioral Control in Human Orbital and Ventral Prefrontal Cortices , 2003, The Journal of Neuroscience.

[30]  Christopher S. Monk,et al.  Choice selection and reward anticipation: an fMRI study , 2004, Neuropsychologia.

[31]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[32]  R. Kikinis,et al.  MRI study of caudate nucleus volume and its cognitive correlates in neuroleptic-naive patients with schizotypal personality disorder. , 2002, The American journal of psychiatry.

[33]  P. Janak,et al.  Posterior dorsomedial striatum is critical for both selective instrumental and Pavlovian reward learning , 2010, The European journal of neuroscience.