Common Neural Mechanisms Underlying Reversal Learning by Reward and Punishment

Impairments in flexible goal-directed decisions, often examined by reversal learning, are associated with behavioral abnormalities characterized by impulsiveness and disinhibition. Although the lateral orbital frontal cortex (OFC) has been consistently implicated in reversal learning, it is still unclear whether this region is involved in negative feedback processing, behavioral control, or both, and whether reward and punishment might have different effects on lateral OFC involvement. Using a relatively large sample (N = 47), and a categorical learning task with either monetary reward or moderate electric shock as feedback, we found overlapping activations in the right lateral OFC (and adjacent insula) for reward and punishment reversal learning when comparing correct reversal trials with correct acquisition trials, whereas we found overlapping activations in the right dorsolateral prefrontal cortex (DLPFC) when negative feedback signaled contingency change. The right lateral OFC and DLPFC also showed greater sensitivity to punishment than did their left homologues, indicating an asymmetry in how punishment is processed. We propose that the right lateral OFC and anterior insula are important for transforming affective feedback to behavioral adjustment, whereas the right DLPFC is involved in higher level attention control. These results provide insight into the neural mechanisms of reversal learning and behavioral flexibility, which can be leveraged to understand risky behaviors among vulnerable populations.

[1]  D. Pine,et al.  The contribution of ventrolateral and dorsolateral prefrontal cortex to response reversal , 2008, Behavioural Brain Research.

[2]  Mark W. Woolrich,et al.  Robust group analysis using outlier inference , 2008, NeuroImage.

[3]  Mark W. Woolrich,et al.  Multilevel linear modelling for FMRI group analysis using Bayesian inference , 2004, NeuroImage.

[4]  Trevor W Robbins,et al.  Lesions of the Medial Striatum in Monkeys Produce Perseverative Impairments during Reversal Learning Similar to Those Produced by Lesions of the Orbitofrontal Cortex , 2008, The Journal of Neuroscience.

[5]  M. Farah,et al.  Ventromedial frontal cortex mediates affective shifting in humans: evidence from a reversal learning paradigm. , 2003, Brain : a journal of neurology.

[6]  T. Robbins,et al.  Defining the Neural Mechanisms of Probabilistic Reversal Learning Using Event-Related Functional Magnetic Resonance Imaging , 2002, The Journal of Neuroscience.

[7]  Mara Mather,et al.  Differential Brain Activity during Emotional versus Nonemotional Reversal Learning , 2012, Journal of Cognitive Neuroscience.

[8]  K I Bolla,et al.  Cerebral Cortex Advance Access published May 13, 2004 Sex-related Differences in a Gambling Task and Its Neurological Correlates , 2022 .

[9]  G. Kok,et al.  Sexual risk behavior among HIV-positive men who have sex with men: a literature review. , 2007, Patient education and counseling.

[10]  Antonio Damasio,et al.  The somatic marker hypothesis: A neural theory of economic decision , 2005, Games Econ. Behav..

[11]  R. Blair,et al.  Response reversal and children with psychopathic tendencies: success is a function of salience of contingency change. , 2005, Journal of child psychology and psychiatry, and allied disciplines.

[12]  H. Uylings,et al.  Reduced orbitofrontal-striatal activity on a reversal learning task in obsessive-compulsive disorder. , 2006, Archives of general psychiatry.

[13]  R. Blair,et al.  Risky decisions and response reversal: is there evidence of orbitofrontal cortex dysfunction in psychopathic individuals? , 2002, Neuropsychologia.

[14]  A M Dale,et al.  Optimal experimental design for event‐related fMRI , 1999, Human brain mapping.

[15]  Dick J. Veltman,et al.  Neural correlates of a reversal learning task with an affectively neutral baseline: An event-related fMRI study , 2005, NeuroImage.

[16]  T. Robbins,et al.  Dissociation in prefrontal cortex of affective and attentional shifts , 1996, Nature.

[17]  R. Valdiserri,et al.  The reemerging HIV/AIDS epidemic in men who have sex with men. , 2007, JAMA.

[18]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[19]  Adrian M. Owen,et al.  Dissociable roles for lateral orbitofrontal cortex and lateral prefrontal cortex during preference driven reversal learning , 2012, NeuroImage.

[20]  T. Robbins,et al.  Contrasting Cortical and Subcortical Activations Produced by Attentional-Set Shifting and Reversal Learning in Humans , 2000, Journal of Cognitive Neuroscience.

[21]  V. Menon,et al.  A critical role for the right fronto-insular cortex in switching between central-executive and default-mode networks , 2008, Proceedings of the National Academy of Sciences.

[22]  J. Gläscher,et al.  Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. , 2009, Cerebral cortex.

[23]  R. Poldrack,et al.  Common neural substrates for inhibition of spoken and manual responses. , 2008, Cerebral cortex.

[24]  W. Overman,et al.  Adult sex differences on a decision-making task previously shown to depend on the orbital prefrontal cortex. , 2001, Behavioral neuroscience.

[25]  V. Menon,et al.  Saliency, switching, attention and control: a network model of insula function , 2010, Brain Structure and Function.

[26]  T. Robbins,et al.  Inhibition and the right inferior frontal cortex , 2004, Trends in Cognitive Sciences.

[27]  L. Fellows,et al.  Beyond Reversal: A Critical Role for Human Orbitofrontal Cortex in Flexible Learning from Probabilistic Feedback , 2010, The Journal of Neuroscience.

[28]  Stephen M. Smith,et al.  General multilevel linear modeling for group analysis in FMRI , 2003, NeuroImage.

[29]  Daniel Tranel,et al.  Executive control deficits in substance-dependent individuals: A comparison of alcohol, cocaine, and methamphetamine and of men and women , 2009, Journal of clinical and experimental neuropsychology.

[30]  Russell A Poldrack,et al.  Neural components underlying behavioral flexibility in human reversal learning. , 2010, Cerebral cortex.

[31]  E. Rolls,et al.  The functional neuroanatomy of the human orbitofrontal cortex: evidence from neuroimaging and neuropsychology , 2004, Progress in Neurobiology.

[32]  Adrian M. Owen,et al.  Inefficiency in Self-organized Attentional Switching in the Normal Aging Population is Associated with Decreased Activity in the Ventrolateral Prefrontal Cortex , 2008, Journal of Cognitive Neuroscience.

[33]  K. Christoff,et al.  Experience sampling during fMRI reveals default network and executive system contributions to mind wandering , 2009, Proceedings of the National Academy of Sciences.

[34]  E. Rolls,et al.  The orbitofrontal cortex and beyond: From affect to decision-making , 2008, Progress in Neurobiology.

[35]  E. Leibenluft,et al.  Impaired probabilistic reversal learning in youths with mood and anxiety disorders , 2009, Psychological Medicine.

[36]  B. Balleine,et al.  Calculating Consequences: Brain Systems That Encode the Causal Effects of Actions , 2008, The Journal of Neuroscience.

[37]  E. Leibenluft,et al.  Abnormal ventromedial prefrontal cortex function in children with psychopathic traits during reversal learning. , 2008, Archives of general psychiatry.

[38]  Jesper Andersson,et al.  Valid conjunction inference with the minimum statistic , 2005, NeuroImage.

[39]  R. Davidson,et al.  Anterior brain electrical asymmetries in response to reward and punishment. , 1992, Electroencephalography and clinical neurophysiology.

[40]  Russell A. Poldrack,et al.  Spaced Learning Enhances Subsequent Recognition Memory by Reducing Neural Repetition Suppression , 2011, Journal of Cognitive Neuroscience.

[41]  P. Andreason,et al.  Gender-related differences in regional cerebral glucose metabolism in normal volunteers , 1994, Psychiatry Research.

[42]  A. Bechara,et al.  Cerebral Cortex doi:10.1093/cercor/bhn147 Functional Dissociations of Risk and Reward Processing in the Medial Prefrontal Cortex , 2008 .

[43]  A. Craig Forebrain emotional asymmetry: a neuroanatomical basis? , 2005, Trends in Cognitive Sciences.

[44]  Sabrina M. Tom,et al.  The Neural Basis of Loss Aversion in Decision-Making Under Risk , 2007, Science.

[45]  J. Hietala,et al.  Sex differences in striatal presynaptic dopamine synthesis capacity in healthy subjects , 2002, Biological Psychiatry.

[46]  M. Mesulam,et al.  Insula of the old world monkey. III: Efferent cortical output and comments on function , 1982, The Journal of comparative neurology.

[47]  R. Poldrack,et al.  Neural Substrates for Reversing Stimulus–Outcome and Stimulus–Response Associations , 2008, The Journal of Neuroscience.

[48]  J. Gläscher,et al.  Dissociable Systems for Gain- and Loss-Related Value Predictions and Errors of Prediction in the Human Brain , 2006, The Journal of Neuroscience.

[49]  R. James R. Blair,et al.  Neural correlates of response reversal: Considering acquisition , 2007, NeuroImage.

[50]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[51]  Scott T. Grafton,et al.  Response to Comment on "Wandering Minds: The Default Network and Stimulus-Independent Thought" , 2007, Science.

[52]  M. Mishkin,et al.  Perseverative interference in monkeys following selective lesions of the inferior prefrontal convexity , 1970, Experimental Brain Research.

[53]  E. Murray,et al.  Bilateral Orbital Prefrontal Cortex Lesions in Rhesus Monkeys Disrupt Choices Guided by Both Reward Value and Reward Contingency , 2004, The Journal of Neuroscience.

[54]  Derek G. V. Mitchell,et al.  Parsing decision making processes in prefrontal cortex: Response inhibition, overcoming learned avoidance, and reversal learning , 2011, NeuroImage.

[55]  Roshan Cools,et al.  Dissociable responses to punishment in distinct striatal regions during reversal learning , 2010, NeuroImage.

[56]  R. Blair,et al.  Adapting to Dynamic Stimulus-Response Values: Differential Contributions of Inferior Frontal, Dorsomedial, and Dorsolateral Regions of Prefrontal Cortex to Decision Making , 2009, The Journal of Neuroscience.

[57]  E. Stein,et al.  Right hemispheric dominance of inhibitory control: an event-related functional MRI study. , 1999, Proceedings of the National Academy of Sciences of the United States of America.

[58]  Lesley K Fellows,et al.  The human ventromedial frontal lobe is critical for learning from negative feedback. , 2008, Brain : a journal of neurology.

[59]  J. O'Doherty,et al.  Decoding the neural substrates of reward-related decision making with functional MRI , 2007, Proceedings of the National Academy of Sciences.

[60]  Vivian V. Valentin,et al.  Overlapping prediction errors in dorsal striatum during instrumental learning with juice and money reward in the human brain. , 2009, Journal of neurophysiology.

[61]  Stephen M. Smith,et al.  A global optimisation method for robust affine registration of brain images , 2001, Medical Image Anal..

[62]  J. O'Doherty,et al.  Dissociating Valence of Outcome from Behavioral Control in Human Orbital and Ventral Prefrontal Cortices , 2003, The Journal of Neuroscience.

[63]  J. O'Doherty,et al.  Overlapping responses for the expectation of juice and money rewards in human ventromedial prefrontal cortex. , 2011, Cerebral cortex.

[64]  R. Blair,et al.  Impaired reversal but intact acquisition: probabilistic response reversal deficits in adult individuals with psychopathy. , 2006, Journal of abnormal psychology.

[65]  M. Farah,et al.  Different underlying impairments in decision-making following ventromedial and dorsolateral frontal lobe damage in humans. , 2004, Cerebral cortex.

[66]  A. Craig How do you feel? Interoception: the sense of the physiological condition of the body , 2002, Nature Reviews Neuroscience.

[67]  W. Overman,et al.  Sex differences in early childhood, adolescence, and adulthood on cognitive tasks that rely on orbital prefrontal cortex , 2004, Brain and Cognition.

[68]  T. Robbins,et al.  Dissociable Contributions of the Orbitofrontal and Infralimbic Cortex to Pavlovian Autoshaping and Discrimination Reversal Learning: Further Evidence for the Functional Heterogeneity of the Rodent Frontal Cortex , 2003, The Journal of Neuroscience.

[69]  M. Gluck,et al.  Functional specialization within the striatum along both the dorsal/ventral and anterior/posterior axes during associative learning via reward and punishment. , 2011, Learning & memory.

[70]  R. Blair,et al.  Divergent Patterns of Aggressive and Neurocognitive Characteristics in Acquired Versus Developmental Psychopathy , 2006, Neurocase.

[71]  H. Fukuyama,et al.  Dissociable mechanisms of attentional control within the human prefrontal cortex. , 2001, Cerebral cortex.

[72]  H. Damasio,et al.  Characterization of the decision-making deficit of patients with ventromedial prefrontal cortex lesions. , 2000, Brain : a journal of neurology.

[73]  T. Robbins,et al.  Orbitofrontal Dysfunction in Patients with Obsessive-Compulsive Disorder and Their Unaffected Relatives , 2008, Science.

[74]  M. Pessiglione,et al.  Critical Roles for Anterior Insula and Dorsal Striatum in Punishment-Based Avoidance Learning , 2012, Neuron.

[75]  R. Cools Dopaminergic modulation of cognitive function-implications for l-DOPA treatment in Parkinson's disease , 2006, Neuroscience & Biobehavioral Reviews.

[76]  E. Rolls,et al.  Abstract reward and punishment representations in the human orbitofrontal cortex , 2001, Nature Neuroscience.

[77]  A. Owen,et al.  Fractionating attentional control using event-related fMRI. , 2005, Cerebral cortex.

[78]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[79]  E. Rolls,et al.  Reward-related Reversal Learning after Surgical Excisions in Orbito-frontal or Dorsolateral Prefrontal Cortex in Humans , 2004, Journal of Cognitive Neuroscience.

[80]  D. V. van Essen,et al.  A Population-Average, Landmark- and Surface-based (PALS) atlas of human cerebral cortex. , 2005, NeuroImage.

[81]  M. Frank,et al.  Striatal Dopamine Predicts Outcome-Specific Reversal Learning and Its Sensitivity to Dopaminergic Drug Administration , 2009, The Journal of Neuroscience.

[82]  P. Dayan,et al.  Differential Encoding of Losses and Gains in the Human Striatum , 2007, The Journal of Neuroscience.

[83]  E. Rolls,et al.  Emotion-related learning in patients with social and emotional changes associated with frontal lobe damage. , 1994, Journal of neurology, neurosurgery, and psychiatry.

[84]  Scott A. Huettel,et al.  Neural Substrates of Contingency Learning and Executive Control: Dissociating Physical, Valuative, and Behavioral Changes , 2009, Front. Hum. Neurosci..

[85]  M. Jenkinson Non-linear registration aka Spatial normalisation , 2007 .

[86]  Geoffrey Schoenbaum,et al.  Orbitofrontal lesions in rats impair reversal but not acquisition of go, no-go odor discriminations , 2002, Neuroreport.