Neural components underlying behavioral flexibility in human reversal learning.

The ability to flexibly respond to changes in the environment is critical for adaptive behavior. Reversal learning (RL) procedures test adaptive response updating when contingencies are altered. We used functional magnetic resonance imaging to examine brain areas that support specific RL components. We compared neural responses to RL and initial learning (acquisition) to isolate reversal-related brain activation independent of cognitive control processes invoked during initial feedback-based learning. Lateral orbitofrontal cortex (OFC) was more activated during reversal than acquisition, suggesting its relevance for reformation of established stimulus-response associations. In addition, the dorsal anterior cingulate (dACC) and right inferior frontal gyrus (rIFG) correlated with change in postreversal accuracy. Because optimal RL likely requires suppression of a prior learned response, we hypothesized that similar regions serve both response inhibition (RI) and inhibition of learned associations during reversal. However, reversal-specific responding and stopping (requiring RI and assessed via the stop-signal task) revealed distinct frontal regions. Although RI-related regions do not appear to support inhibition of prepotent learned associations, a subset of these regions, dACC and rIFG, guide actions consistent with current reward contingencies. These regions and lateral OFC represent distinct neural components that support behavioral flexibility important for adaptive learning.

[1]  M. Frank,et al.  Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. , 2006, Psychological review.

[2]  Michael Brady,et al.  Improved Optimization for the Robust and Accurate Linear Registration and Motion Correction of Brain Images , 2002, NeuroImage.

[3]  Mark W. Woolrich,et al.  Multilevel linear modelling for FMRI group analysis using Bayesian inference , 2004, NeuroImage.

[4]  W. Schultz,et al.  Modifications of reward expectation-related neuronal activity during learning in primate orbitofrontal cortex. , 2000, Journal of neurophysiology.

[5]  T. Robbins,et al.  Orbitofrontal Dysfunction in Patients with Obsessive-Compulsive Disorder and Their Unaffected Relatives , 2008, Science.

[6]  M. Petrides Deficits on conditional associative-learning tasks after frontal- and temporal-lobe lesions in man , 1985, Neuropsychologia.

[7]  N. Volkow,et al.  The orbitofrontal cortex in drug addiction , 2006 .

[8]  Trevor W Robbins,et al.  Lesions of the Medial Striatum in Monkeys Produce Perseverative Impairments during Reversal Learning Similar to Those Produced by Lesions of the Orbitofrontal Cortex , 2008, The Journal of Neuroscience.

[9]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[10]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[11]  R. Poldrack,et al.  Neural Substrates for Reversing Stimulus–Outcome and Stimulus–Response Associations , 2008, The Journal of Neuroscience.

[12]  E. Rolls,et al.  Abstract reward and punishment representations in the human orbitofrontal cortex , 2001, Nature Neuroscience.

[13]  R. James R. Blair,et al.  Neural correlates of response reversal: Considering acquisition , 2007, NeuroImage.

[14]  Jesper Andersson,et al.  Valid conjunction inference with the minimum statistic , 2005, NeuroImage.

[15]  T. Robbins,et al.  Inhibition and the right inferior frontal cortex , 2004, Trends in Cognitive Sciences.

[16]  Peter Kirsch,et al.  On Framing Effects in Decision Making: Linking Lateral versus Medial Orbitofrontal Cortex Activation to Choice Outcome Processing , 2006, Journal of Cognitive Neuroscience.

[17]  Stephen M. Smith,et al.  General multilevel linear modeling for group analysis in FMRI , 2003, NeuroImage.

[18]  Lesley K Fellows,et al.  The human ventromedial frontal lobe is critical for learning from negative feedback. , 2008, Brain : a journal of neurology.

[19]  H. Duvernoy,et al.  The Human Brain: Surface, Three-Dimensional Sectional Anatomy with MRI, and Blood Supply , 1999 .

[20]  M. Roesch,et al.  The Orbitofrontal Cortex and Ventral Tegmental Area Are Necessary for Learning from Unexpected Outcomes , 2009, Neuron.

[21]  Anders M. Dale,et al.  Reliability in multi-site structural MRI studies: Effects of gradient non-linearity correction on phantom and human data , 2006, NeuroImage.

[22]  R. Poldrack,et al.  Cortical and Subcortical Contributions to Stop Signal Response Inhibition: Role of the Subthalamic Nucleus , 2006, The Journal of Neuroscience.

[23]  Arthur W. Toga,et al.  Automatic independent component labeling for artifact removal in fMRI , 2008, NeuroImage.

[24]  P. Montague,et al.  Activity in human ventral striatum locked to errors of reward prediction , 2002, Nature Neuroscience.

[25]  J. Pujol,et al.  Mapping structural brain alterations in obsessive-compulsive disorder. , 2004, Archives of general psychiatry.

[26]  Timothy E. J. Behrens,et al.  Frontal Cortex Subregions Play Distinct Roles in Choices between Actions and Stimuli , 2008, The Journal of Neuroscience.

[27]  J. O'Doherty,et al.  Dissociating Valence of Outcome from Behavioral Control in Human Orbital and Ventral Prefrontal Cortices , 2003, The Journal of Neuroscience.

[28]  M. Petrides Visuo-motor conditional associative learning after frontal and temporal lesions in the human brain , 1997, Neuropsychologia.

[29]  P. Strick,et al.  Imaging the premotor areas , 2001, Current Opinion in Neurobiology.

[30]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[31]  W. Schultz Getting Formal with Dopamine and Reward , 2002, Neuron.

[32]  M. Reuter,et al.  Dopamine DRD2 polymorphism alters reversal learning and associated neural activity , 2009, NeuroImage.

[33]  R. Dolan,et al.  Subliminal Instrumental Conditioning Demonstrated in the Human Brain , 2008, Neuron.

[34]  M. Mishkin,et al.  Perseverative interference in monkeys following selective lesions of the inferior prefrontal convexity , 1970, Experimental Brain Research.

[35]  R. Elliott,et al.  Dissociable functions in the medial and lateral orbitofrontal cortex: evidence from human neuroimaging studies. , 2000, Cerebral cortex.

[36]  E. Murray,et al.  Bilateral Orbital Prefrontal Cortex Lesions in Rhesus Monkeys Disrupt Choices Guided by Both Reward Value and Reward Contingency , 2004, The Journal of Neuroscience.

[37]  Timothy Edward John Behrens,et al.  Triangulating a Cognitive Control Network Using Diffusion-Weighted Magnetic Resonance Imaging (MRI) and Functional MRI , 2007, The Journal of Neuroscience.

[38]  T. Robbins,et al.  Dissociation in prefrontal cortex of affective and attentional shifts , 1996, Nature.

[39]  M. Farah,et al.  Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. , 2003, Brain : a journal of neurology.

[40]  Charles M. Butter,et al.  Perseveration in extinction and in discrimination reversal tasks following selective frontal ablations in Macaca mulatta , 1969 .

[41]  L. Fellows The Role of Orbitofrontal Cortex in Decision Making , 2007, Annals of the New York Academy of Sciences.

[42]  J. Parkinson,et al.  Dissociable Contributions of the Human Amygdala and Orbitofrontal Cortex to Incentive Motivation and Goal Selection , 2003, The Journal of Neuroscience.

[43]  Christian Bellebaum,et al.  Focal basal ganglia lesions are associated with impairments in reward-based reversal learning. , 2008, Brain : a journal of neurology.

[44]  Stephen M. Smith,et al.  Temporal Autocorrelation in Univariate Linear Modeling of FMRI Data , 2001, NeuroImage.

[45]  H. Uylings,et al.  Reduced orbitofrontal-striatal activity on a reversal learning task in obsessive-compulsive disorder. , 2006, Archives of general psychiatry.

[46]  M. D’Esposito,et al.  Reversal learning in Parkinson's disease depends on medication status and outcome valence , 2006, Neuropsychologia.

[47]  Alan C. Evans,et al.  A Three-Dimensional Statistical Analysis for CBF Activation Studies in Human Brain , 1992, Journal of cerebral blood flow and metabolism : official journal of the International Society of Cerebral Blood Flow and Metabolism.

[48]  T. Robbins,et al.  Defining the Neural Mechanisms of Probabilistic Reversal Learning Using Event-Related Functional Magnetic Resonance Imaging , 2002, The Journal of Neuroscience.

[49]  Morten L Kringelbach,et al.  Neural correlates of rapid reversal learning in a simple model of human social interaction , 2003, NeuroImage.

[50]  M. Mishkin,et al.  Limbic lesions and the problem of stimulus--reinforcement associations. , 1972, Experimental neurology.

[51]  Dick J. Veltman,et al.  Neural correlates of a reversal learning task with an affectively neutral baseline: An event-related fMRI study , 2005, NeuroImage.

[52]  J. O'Doherty,et al.  What We Know and Do Not Know about the Functions of the Orbitofrontal Cortex after 20 Years of Cross-Species Studies , 2007, The Journal of Neuroscience.

[53]  Samuel M. McClure,et al.  BOLD Responses Reflecting Dopaminergic Signals in the Human Ventral Tegmental Area , 2008, Science.

[54]  Laura Bellodi,et al.  Decision-making heterogeneity in obsessive-compulsive disorder: ventromedial prefrontal cortex function predicts different treatment outcomes , 2002, Neuropsychologia.

[55]  Geoffrey Schoenbaum,et al.  Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations , 2002, Neuroreport.

[56]  E. Rolls,et al.  Reward-related Reversal Learning after Surgical Excisions in Orbito-frontal or Dorsolateral Prefrontal Cortex in Humans , 2004, Journal of Cognitive Neuroscience.

[57]  K. Grill-Spector,et al.  Repetition and the brain: neural models of stimulus-specific effects , 2006, Trends in Cognitive Sciences.

[58]  N. White Mnemonic functions of the basal ganglia , 1997, Current Opinion in Neurobiology.

[59]  M. Frank,et al.  Striatal Dopamine Predicts Outcome-Specific Reversal Learning and Its Sensitivity to Dopaminergic Drug Administration , 2009, The Journal of Neuroscience.

[60]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[61]  M. Ragozzino The Contribution of the Medial Prefrontal Cortex, Orbitofrontal Cortex, and Dorsomedial Striatum to Behavioral Flexibility , 2007, Annals of the New York Academy of Sciences.