Neural Substrates for Reversing Stimulus–Outcome and Stimulus–Response Associations

Adaptive goal-directed actions require the ability to quickly relearn behaviors in a changing environment, yet how the brain supports this ability is barely understood. Using functional magnetic resonance imaging and a novel reversal learning paradigm, the present study examined the neural mechanisms associated with reversal learning for outcomes versus motor responses. Participants were extensively trained to classify novel visual symbols (Japanese Hiraganas) into two arbitrary classes (“male” or “female”), in which subjects could acquire both stimulus–outcome associations and stimulus–response associations. They were then required to relearn either the outcome or the motor response associated with the symbols, or both. The results revealed that during reversal learning, a network including anterior cingulate, posterior inferior frontal, and parietal regions showed extended activation for all types of reversal trials, whereas their activation decreased quickly for trials not involving reversal, suggesting their role in domain–general interference resolution. The later increase of right ventral lateral prefrontal cortex and caudate for reversal of stimulus–outcome associations suggests their importance in outcome reversal learning in the face of interference.

[1]  R. C. Oldfield The assessment and analysis of handedness: the Edinburgh inventory. , 1971, Neuropsychologia.

[2]  T. Robbins,et al.  Dissociation in prefrontal cortex of affective and attentional shifts , 1996, Nature.

[3]  M. Farah,et al.  Role of left inferior prefrontal cortex in retrieval of semantic knowledge: a reevaluation. , 1997, Proceedings of the National Academy of Sciences of the United States of America.

[4]  M. Botvinick,et al.  Anterior cingulate cortex, error detection, and the online monitoring of performance. , 1998, Science.

[5]  A. Schleicher,et al.  Broca's region revisited: Cytoarchitecture and intersubject variability , 1999, The Journal of comparative neurology.

[6]  Jonathan D. Cohen,et al.  Conflict monitoring versus selection-for-action in anterior cingulate cortex , 1999, Nature.

[7]  A M Dale,et al.  Optimal experimental design for event‐related fMRI , 1999, Human brain mapping.

[8]  T. Braver,et al.  Anterior Cingulate Cortex and Response Conflict : Effects of Response Modality and Processing Domain , 2022 .

[9]  L. Nystrom,et al.  Tracking the hemodynamic responses to reward and punishment in the striatum. , 2000, Journal of neurophysiology.

[10]  M. Botvinick,et al.  Conflict monitoring and cognitive control. , 2001, Psychological review.

[11]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[12]  M. Gluck,et al.  Interactive memory systems in the human brain , 2001, Nature.

[13]  E. Rolls,et al.  Abstract reward and punishment representations in the human orbitofrontal cortex , 2001, Nature Neuroscience.

[14]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[15]  T. Braver,et al.  Anterior cingulate cortex and response conflict: effects of response modality and processing domain. , 2001, Cerebral Cortex.

[16]  David J. Freedman,et al.  Categorical representation of visual stimuli in the primate prefrontal cortex. , 2001, Science.

[17]  R. Poldrack,et al.  Recovering Meaning Left Prefrontal Cortex Guides Controlled Semantic Retrieval , 2001, Neuron.

[18]  Eliot Hazeltine,et al.  Dissociable Contributions of Prefrontal and Parietal Cortices to Response Selection , 2002, NeuroImage.

[19]  David J. Freedman,et al.  Visual categorization and the primate prefrontal cortex: neurophysiology and behavior. , 2002, Journal of neurophysiology.

[20]  T. Robbins,et al.  Defining the Neural Mechanisms of Probabilistic Reversal Learning Using Event-Related Functional Magnetic Resonance Imaging , 2002, The Journal of Neuroscience.

[21]  Stephen M. Smith,et al.  General multilevel linear modeling for group analysis in FMRI , 2003, NeuroImage.

[22]  J. O'Doherty,et al.  Dissociating Valence of Outcome from Behavioral Control in Human Orbital and Ventral Prefrontal Cortices , 2003, The Journal of Neuroscience.

[23]  G. Pagnoni,et al.  Human Striatal Response to Salient Nonrewarding Stimuli , 2003, The Journal of Neuroscience.

[24]  T. Robbins,et al.  Inhibition and the right inferior frontal cortex , 2004, Trends in Cognitive Sciences.

[25]  M. Walton,et al.  Action sets and decisions in the medial frontal cortex , 2004, Trends in Cognitive Sciences.

[26]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[27]  K. R. Ridderinkhof,et al.  Neurocognitive mechanisms of cognitive control: The role of prefrontal cortex in action selection, response inhibition, performance monitoring, and reward-based learning , 2004, Brain and Cognition.

[28]  Jan Derrfuss,et al.  Cognitive control in the posterior frontolateral cortex: evidence from common activations in task coordination, interference control, and working memory , 2004, NeuroImage.

[29]  Mark W. Woolrich,et al.  Multilevel linear modelling for FMRI group analysis using Bayesian inference , 2004, NeuroImage.

[30]  K. R. Ridderinkhof,et al.  The Role of the Medial Frontal Cortex in Cognitive Control , 2004, Science.

[31]  Jonathan D. Cohen,et al.  Anterior Cingulate Conflict Monitoring and Adjustments in Control , 2004, Science.

[32]  E. Rolls,et al.  Reward-related Reversal Learning after Surgical Excisions in Orbito-frontal or Dorsolateral Prefrontal Cortex in Humans , 2004, Journal of Cognitive Neuroscience.

[33]  M. Delgado,et al.  Modulation of Caudate Activity by Action Contingency , 2004, Neuron.

[34]  M. Mishkin,et al.  Perseverative interference in monkeys following selective lesions of the inferior prefrontal convexity , 1970, Experimental Brain Research.

[35]  E. Miller,et al.  Different time courses of learning-related activity in the prefrontal cortex and striatum , 2005, Nature.

[36]  Joshua W. Brown,et al.  Learned Predictions of Error Likelihood in the Anterior Cingulate Cortex , 2005, Science.

[37]  M. Petrides Lateral prefrontal cortex: architectonic and functional organization , 2005, Philosophical Transactions of the Royal Society B: Biological Sciences.

[38]  Carol A. Seger,et al.  The Roles of the Caudate Nucleus in Human Classification Learning , 2005, The Journal of Neuroscience.

[39]  Dick J. Veltman,et al.  Neural correlates of a reversal learning task with an affectively neutral baseline: An event-related fMRI study , 2005, NeuroImage.

[40]  Mark Laubach,et al.  Who's on first? What's on second? The time course of learning in corticostriatal systems , 2005, Trends in Neurosciences.

[41]  B. Balleine,et al.  The role of the dorsomedial striatum in instrumental conditioning , 2005, The European journal of neuroscience.

[42]  David Badre,et al.  Frontal lobe mechanisms that resolve proactive interference. , 2005, Cerebral cortex.

[43]  Jesper Andersson,et al.  Valid conjunction inference with the minimum statistic , 2005, NeuroImage.

[44]  R. Poldrack,et al.  Dissociable Controlled Retrieval and Generalized Selection Mechanisms in Ventrolateral Prefrontal Cortex , 2005, Neuron.

[45]  M. Brass,et al.  Involvement of the inferior frontal junction in cognitive control: Meta‐analyses of switching and Stroop studies , 2005, Human brain mapping.

[46]  B. Balleine,et al.  Blockade of NMDA receptors in the dorsomedial striatum prevents action–outcome learning in instrumental conditioning , 2005, The European journal of neuroscience.

[47]  David Badre,et al.  Computational and neurobiological mechanisms underlying cognitive flexibility. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[48]  H. Yin,et al.  The role of the basal ganglia in habit formation , 2006, Nature Reviews Neuroscience.

[49]  L. Shiu,et al.  Unlearning a stimulus–response association , 2006, Psychological research.

[50]  Giulio Tononi,et al.  Direct evidence for a prefrontal contribution to the control of proactive interference in verbal working memory , 2006, Proceedings of the National Academy of Sciences.

[51]  Christopher L. Asplund,et al.  Isolation of a Central Bottleneck of Information Processing with Time-Resolved fMRI , 2006, Neuron.

[52]  R. Poldrack,et al.  Cortical and Subcortical Contributions to Stop Signal Response Inhibition: Role of the Subthalamic Nucleus , 2006, The Journal of Neuroscience.

[53]  R. James R. Blair,et al.  Neural correlates of response reversal: Considering acquisition , 2007, NeuroImage.

[54]  Jeffrey D. Karpicke,et al.  Expanding retrieval practice promotes short-term retention, but equally spaced retrieval enhances long-term retention. , 2007, Journal of experimental psychology. Learning, memory, and cognition.

[55]  Michael J. Frank,et al.  Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning , 2007, Proceedings of the National Academy of Sciences.

[56]  Timothy Edward John Behrens,et al.  Triangulating a Cognitive Control Network Using Diffusion-Weighted Magnetic Resonance Imaging (MRI) and Functional MRI , 2007, The Journal of Neuroscience.

[57]  R. Poldrack,et al.  Common neural substrates for inhibition of spoken and manual responses. , 2008, Cerebral cortex.

[58]  Arthur W. Toga,et al.  Automatic independent component labeling for artifact removal in fMRI , 2008, NeuroImage.