Neural Substrates of Contingency Learning and Executive Control: Dissociating Physical, Valuative, and Behavioral Changes

Contingency learning is fundamental to cognition. Knowledge about environmental contingencies allows behavioral flexibility, as executive control processes accommodate the demands of novel or changing environments. Studies of experiential learning have focused on the relationship between actions and the values of associated outcomes. However, outcome values have often been confounded with the physical changes in the outcomes themselves. Here, we dissociated contingency learning into valuative and non-valuative forms, using a novel version of the two-alternative choice task, while measuring the neural effects of contingency changes using functional magnetic resonance imaging (fMRI). Changes in value-relevant contingencies evoked activation in the lateral prefrontal cortex (LPFC), posterior parietal cortex (PPC), and dorsomedial prefrontal cortex (DMPFC) consistent with prior results (e.g., reversal-learning paradigms). Changes in physical contingencies unrelated to value or to action produced similar activations within the LPFC, indicating that LPFC may engage in generalized contingency learning that is not specific to valuation. In contrast, contingency changes that required behavioral shifts evoked activation localized to the DMPFC, supplementary motor, and precentral cortices, suggesting that these regions play more specific roles within the executive control of behavior.

[1]  P. Glimcher,et al.  Activity in Posterior Parietal Cortex Is Correlated with the Relative Subjective Desirability of Action , 2004, Neuron.

[2]  S. Finger,et al.  Tactile-visual acquisition and reversal learning deficits in rats with prefrontal cortical lesions , 2004, Experimental Brain Research.

[3]  R. Poldrack,et al.  Neural Substrates for Reversing Stimulus–Outcome and Stimulus–Response Associations , 2008, The Journal of Neuroscience.

[4]  R. Passingham,et al.  The prefrontal cortex: response selection or maintenance within working memory? , 2000, 5th IEEE EMBS International Summer School on Biomedical Imaging, 2002..

[5]  E. Thorndike SOME EXPERIMENTS ON ANIMAL INTELLIGENCE. , 1898, Science.

[6]  P. Goldman-Rakic,et al.  Infrequent events transiently activate human prefrontal and parietal cortex as measured by functional MRI. , 1997, Journal of neurophysiology.

[7]  N. Squires,et al.  The effect of stimulus sequence on the waveform of the cortical event-related potential. , 1976, Science.

[8]  Lawrence H Snyder,et al.  Single Neurons in Posterior Parietal Cortex of Monkeys Encode Cognitive Set , 2004, Neuron.

[9]  C. Summerfield,et al.  Two Mechanisms for Task Switching in the Prefrontal Cortex , 2009, The Journal of Neuroscience.

[10]  R. Andersen,et al.  Reaches to Sounds Encoded in an Eye-Centered Reference Frame , 2000, Neuron.

[11]  M. Shadlen,et al.  Decision-making with multiple alternatives , 2008, Nature Neuroscience.

[12]  R. Andersen,et al.  Intention-related activity in the posterior parietal cortex: a review , 2000, Vision Research.

[13]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[14]  P. Goldman-Rakic,et al.  Matching patterns of activity in primate prefrontal area 8a and parietal area 7ip neurons during a spatial working memory task. , 1998, Journal of neurophysiology.

[15]  Mark W. Woolrich,et al.  Bayesian analysis of neuroimaging data in FSL , 2009, NeuroImage.

[16]  Richard Passingham,et al.  Delayed matching after selective prefrontal lesions in monkeys (Macaca mulatta) , 1975, Brain Research.

[17]  Jörn Diedrichsen,et al.  Neural substrates of visuomotor learning based on improved feedback control and prediction , 2008, NeuroImage.

[18]  Chris Rorden,et al.  Improving Lesion-Symptom Mapping , 2007, Journal of Cognitive Neuroscience.

[19]  E. Rolls,et al.  Abstract reward and punishment representations in the human orbitofrontal cortex , 2001, Nature Neuroscience.

[20]  Francisco Barceló,et al.  An Information Theoretical Approach to Task-Switching: Evidence from Cognitive Brain Potentials in Humans , 2007, Frontiers in human neuroscience.

[21]  G. McCarthy,et al.  Perceiving patterns in random series: dynamic processing of sequence in prefrontal cortex , 2002, Nature Neuroscience.

[22]  R. Herrnstein On the law of effect. , 1970, Journal of the experimental analysis of behavior.

[23]  Raymond J. Dolan,et al.  Contingency awareness in human aversive conditioning involves the middle frontal gyrus , 2006, NeuroImage.

[24]  P. Goldman-Rakic The prefrontal landscape: implications of functional architecture for understanding human mentation and the central executive. , 1996, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[25]  R. M. Elliott,et al.  Behavior of Organisms , 1991 .

[26]  T. Robbins,et al.  Extra-dimensional versus intra-dimensional set shifting performance following frontal lobe excisions, temporal lobe excisions or amygdalo-hippocampectomy in man , 1991, Neuropsychologia.

[27]  Michael L. Platt,et al.  Neural correlates of decision variables in parietal cortex , 1999, Nature.

[28]  Guy Vingerhoets,et al.  Knowing about tools: Neural correlates of tool familiarity and experience , 2008, NeuroImage.

[29]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[30]  Mark W. Woolrich,et al.  Advances in functional and structural MR image analysis and implementation as FSL , 2004, NeuroImage.

[31]  Colin Camerer,et al.  Dissociating the Role of the Orbitofrontal Cortex and the Striatum in the Computation of Goal Values and Prediction Errors , 2008, The Journal of Neuroscience.

[32]  R. J. Dolan,et al.  Differential neural response to positive and negative feedback in planning and guessing tasks , 1997, Neuropsychologia.

[33]  Bruce D. McCandliss,et al.  Response Anticipation and Response Conflict: An Event-Related Potential and Functional Magnetic Resonance Imaging Study , 2007, The Journal of Neuroscience.

[34]  T. Robbins,et al.  Defining the Neural Mechanisms of Probabilistic Reversal Learning Using Event-Related Functional Magnetic Resonance Imaging , 2002, The Journal of Neuroscience.

[35]  E. Murray,et al.  Amygdala and Orbitofrontal Cortex Lesions Differentially Influence Choices during Object Reversal Learning , 2008, The Journal of Neuroscience.

[36]  Dick J. Veltman,et al.  Neural correlates of a reversal learning task with an affectively neutral baseline: An event-related fMRI study , 2005, NeuroImage.

[37]  J. O'Doherty,et al.  Marketing actions can modulate neural representations of experienced pleasantness , 2008, Proceedings of the National Academy of Sciences.

[38]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[39]  J. Cohen,et al.  Context, cortex, and dopamine: a connectionist approach to behavior and biology in schizophrenia. , 1992, Psychological review.

[40]  A. Damasio,et al.  Insensitivity to future consequences following damage to human prefrontal cortex , 1994, Cognition.

[41]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[42]  F. W. Irwin Purposive Behavior in Animals and Men , 1932, The Psychological Clinic.

[43]  E. Murray,et al.  Bilateral Orbital Prefrontal Cortex Lesions in Rhesus Monkeys Disrupt Choices Guided by Both Reward Value and Reward Contingency , 2004, The Journal of Neuroscience.

[44]  M. Hallett,et al.  Induction of a recall deficit by rapid-rate transcranial magnetic stimulation. , 1994, Neuroreport.

[45]  J. Assad,et al.  Neural coding of behavioral relevance in parietal cortex , 2003, Current Opinion in Neurobiology.

[46]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[47]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[48]  V. Menon,et al.  A critical role for the right fronto-insular cortex in switching between central-executive and default-mode networks , 2008, Proceedings of the National Academy of Sciences.

[49]  Matthew T. Kaufman,et al.  Distributed Neural Representation of Expected Value , 2005, The Journal of Neuroscience.

[50]  Joseph E LeDoux,et al.  From Fear to Safety and Back: Reversal of Fear in the Human Brain , 2008, The Journal of Neuroscience.

[51]  Keiji Tanaka,et al.  Conflict-induced behavioural adjustment: a clue to the executive functions of the prefrontal cortex , 2009, Nature Reviews Neuroscience.

[52]  P. Glimcher,et al.  Action and Outcome Encoding in the Primate Caudate Nucleus , 2007, The Journal of Neuroscience.

[53]  Yale E Cohen,et al.  Modulation of LIP activity by predictive auditory and visual cues. , 2004, Cerebral cortex.

[54]  J. Assad,et al.  Dynamic coding of behaviourally relevant stimuli in parietal cortex , 2002, Nature.

[55]  M. Goldberg,et al.  Space and attention in parietal cortex. , 1999, Annual review of neuroscience.

[56]  C. Gerfen,et al.  The frontal cortex-basal ganglia system in primates. , 1996, Critical reviews in neurobiology.

[57]  K. R. Ridderinkhof,et al.  The Role of the Medial Frontal Cortex in Cognitive Control , 2004, Science.

[58]  M. Walton,et al.  Interactions between decision making and performance monitoring within prefrontal cortex , 2004, Nature Neuroscience.

[59]  Mark W. Woolrich,et al.  Robust group analysis using outlier inference , 2008, NeuroImage.

[60]  Stephen M Smith,et al.  Fast robust automated brain extraction , 2002, Human brain mapping.

[61]  Scott A. Huettel,et al.  What is odd in the oddball task? Prefrontal cortex is activated by dynamic changes in response strategy , 2004, Neuropsychologia.

[62]  J. O'Doherty,et al.  Orbitofrontal Cortex Encodes Willingness to Pay in Everyday Economic Transactions , 2007, The Journal of Neuroscience.

[63]  T. Robbins,et al.  Dissociation in prefrontal cortex of affective and attentional shifts , 1996, Nature.

[64]  J. O'Doherty,et al.  Dissociating Valence of Outcome from Behavioral Control in Human Orbital and Ventral Prefrontal Cortices , 2003, The Journal of Neuroscience.

[65]  J. Duncan,et al.  Common regions of the human frontal lobe recruited by diverse cognitive demands , 2000, Trends in Neurosciences.

[66]  E. Rolls,et al.  Reward-related Reversal Learning after Surgical Excisions in Orbito-frontal or Dorsolateral Prefrontal Cortex in Humans , 2004, Journal of Cognitive Neuroscience.

[67]  Alex Martin,et al.  A neural system for learning about object function. , 2006, Cerebral cortex.

[68]  M. Botvinick,et al.  Conflict monitoring and cognitive control. , 2001, Psychological review.

[69]  Timothy D. Hanks,et al.  Probabilistic Population Codes for Bayesian Decision Making , 2008, Neuron.

[70]  Matthias Weisbrod,et al.  Prefrontal-cingulate activation during executive control: which comes first? , 2004, Brain research. Cognitive brain research.

[71]  Y. Cohen,et al.  Eye-centered, head-centered, and complex coding of visual and auditory targets in the intraparietal sulcus. , 2005, Journal of neurophysiology.

[72]  M. Delgado,et al.  Dorsal striatum responses to reward and punishment: Effects of valence and magnitude manipulations , 2003, Cognitive, affective & behavioral neuroscience.

[73]  D. Whitteridge Lectures on Conditioned Reflexes , 1942, Nature.

[74]  Leslie G. Ungerleider,et al.  Neural correlates of category-specific knowledge , 1996, Nature.

[75]  T. Robbins Shifting and stopping: fronto-striatal substrates, neurochemical modulation and clinical implications , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.