Serotonin and Aversive Pavlovian Control of Instrumental Behavior in Humans

Adaptive behavior involves interactions between systems regulating Pavlovian and instrumental control of actions. Here, we present the first investigation of the neural mechanisms underlying aversive Pavlovian–instrumental transfer using fMRI in humans. Recent evidence indicates that these Pavlovian influences on instrumental actions are action-specific: Instrumental approach is invigorated by appetitive Pavlovian cues but inhibited by aversive Pavlovian cues. Conversely, instrumental withdrawal is inhibited by appetitive Pavlovian cues but invigorated by aversive Pavlovian cues. We show that BOLD responses in the amygdala and the nucleus accumbens were associated with behavioral inhibition by aversive Pavlovian cues, irrespective of action context. Furthermore, BOLD responses in the ventromedial prefrontal cortex differed between approach and withdrawal actions. Aversive Pavlovian conditioned stimuli modulated connectivity between the ventromedial prefrontal cortex and the caudate nucleus. These results show that action-specific aversive control of instrumental behavior involves the modulation of fronto-striatal interactions by Pavlovian conditioned stimuli.

[1]  Rupert Lanzenberger,et al.  Serotonin and molecular neuroimaging in humans using PET , 2011, Amino Acids.

[2]  E. Phelps,et al.  Neural mechanisms mediating optimism bias , 2007, Nature.

[3]  D. Cornell,et al.  Psychopathy in instrumental and reactive violent offenders. , 1996, Journal of consulting and clinical psychology.

[4]  B. Balleine,et al.  Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual Action , 2010, Neuropsychopharmacology.

[5]  P. Holland Relations between Pavlovian-instrumental transfer and reinforcer devaluation. , 2004, Journal of experimental psychology. Animal behavior processes.

[6]  Serotonin and dopamine differentially affect appetitive and aversive general Pavlovian-to-instrumental transfer , 2014, Psychopharmacology.

[7]  Roshan Cools,et al.  Opposing Effects of Appetitive and Aversive Cues on Go/No-go Behavior and Motor Excitability , 2014, Journal of Cognitive Neuroscience.

[8]  R. Pihl,et al.  Tryptophan depletion causes a rapid lowering of mood in normal males , 2004, Psychopharmacology.

[9]  Timothy E. J. Behrens,et al.  Review Frontal Cortex and Reward-guided Learning and Decision-making Figure 1. Frontal Brain Regions in the Macaque Involved in Reward-guided Learning and Decision-making Finer Grained Anatomical Divisions with Frontal Cortical Systems for Reward-guided Behavior , 2022 .

[10]  Stephen M. Smith,et al.  A Bayesian model of shape and appearance for subcortical brain segmentation , 2011, NeuroImage.

[11]  G. Kranz,et al.  Reward and the serotonergic system , 2010, Neuroscience.

[12]  B. Sahakian,et al.  Tryptophan depletion disinhibits punishment but not reward prediction: implications for resilience , 2011, Psychopharmacology.

[13]  R. Wurtman,et al.  Brain serotonin content: physiological regulation by plasma neutral amino acids. , 1997, Science.

[14]  Joseph E LeDoux,et al.  Medial Amygdala Lesions Selectively Block Aversive Pavlovian–Instrumental Transfer in Rats , 2014, Front. Behav. Neurosci..

[15]  P. Dayan,et al.  Opponency Revisited: Competition and Cooperation Between Dopamine and Serotonin , 2010, Neuropsychopharmacology.

[16]  J. Monahan,et al.  Rethinking Risk Assessment: The MacArthur Study of Mental Disorder and Violence , 2001 .

[17]  B Schmand,et al.  [The Dutch Reading Test for Adults: a measure of premorbid intelligence level]. , 1991, Tijdschrift voor gerontologie en geriatrie.

[18]  N. Tzourio-Mazoyer,et al.  Automated Anatomical Labeling of Activations in SPM Using a Macroscopic Anatomical Parcellation of the MNI MRI Single-Subject Brain , 2002, NeuroImage.

[19]  P. Dayan,et al.  Action versus valence in decision making , 2014, Trends in Cognitive Sciences.

[20]  Melissa J. Allman,et al.  Learning processes affecting human decision making: An assessment of reinforcer-selective Pavlovian-to-instrumental transfer following reinforcer devaluation. , 2010, Journal of experimental psychology. Animal behavior processes.

[21]  C. Ávila,et al.  The Sensitivity to Punishment and Sensitivity to Reward Questionnaire (SPSRQ) as a measure of Gray's anxiety and impulsivity dimensions. , 2001 .

[22]  Jonathan D. Cohen,et al.  An fMRI Investigation of Emotional Engagement in Moral Judgment , 2001, Science.

[23]  G. E. Alexander,et al.  Parallel organization of functionally segregated circuits linking basal ganglia and cortex. , 1986, Annual review of neuroscience.

[24]  Sylvia M. L. Cox,et al.  Tryptophan Depletion Disrupts the Motivational Guidance of Goal-Directed Behavior as a Function of Trait Impulsivity , 2005, Neuropsychopharmacology.

[25]  B. Balleine,et al.  Double Dissociation of Basolateral and Central Amygdala Lesions on the General and Outcome-Specific Forms of Pavlovian-Instrumental Transfer , 2005, The Journal of Neuroscience.

[26]  A. Young,et al.  Rapid depletion of plasma tryptophan: a review of studies and experimental methodology , 1997, Journal of psychopharmacology.

[27]  A. Tversky,et al.  The framing of decisions and the psychology of choice. , 1981, Science.

[28]  K. Doya,et al.  The Role of Serotonin in the Regulation of Patience and Impulsivity , 2012, Molecular Neurobiology.

[29]  R. Wurtman,et al.  Brain Serotonin Content: Physiological Regulation by Plasma Neutral Amino Acids , 1972, Science.

[30]  T. Robbins,et al.  Serotonin Modulates the Effects of Pavlovian Aversive Predictions on Response Vigor , 2012, Neuropsychopharmacology.

[31]  T. Robbins,et al.  Reconciling the Role of Serotonin in Behavioral Inhibition and Aversion: Acute Tryptophan Depletion Abolishes Punishment-Induced Inhibition in Humans , 2009, The Journal of Neuroscience.

[32]  Q. Huys,et al.  Individual differences in bodily freezing predict emotional biases in decision making , 2014, Front. Behav. Neurosci..

[33]  Peter Dayan,et al.  Non-commercial Research and Educational Use including without Limitation Use in Instruction at Your Institution, Sending It to Specific Colleagues That You Know, and Providing a Copy to Your Institution's Administrator. All Other Uses, Reproduction and Distribution, including without Limitation Comm , 2022 .

[34]  D H Brainard,et al.  The Psychophysics Toolbox. , 1997, Spatial vision.

[35]  Hans J. Eysenck,et al.  Manual of the Eysenck personality questionnaire , 1975 .

[36]  Raymond J. Dolan,et al.  Go and no-go learning in reward and punishment: Interactions between affect and effect , 2012, NeuroImage.

[37]  P. Soubrié Reconciling the role of central serotonin neurons in human and animal behavior , 1986, Behavioral and Brain Sciences.

[38]  A. Damasio The somatic marker hypothesis and the possible functions of the prefrontal cortex. , 1996, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[39]  L. Parsons,et al.  Reciprocal limbic-cortical function and negative mood: converging PET findings in depression and normal sadness. , 1999, The American journal of psychiatry.

[40]  A. Roberts,et al.  Serotonin at the level of the amygdala and orbitofrontal cortex modulates distinct aspects of positive emotion in primates , 2011, The international journal of neuropsychopharmacology.

[41]  J. Trew,et al.  Exploring the roles of approach and avoidance in depression: an integrative model. , 2011, Clinical psychology review.

[42]  Roshan Cools,et al.  Acute serotonin depletion releases motivated inhibition of response vigour , 2014, Psychopharmacology.

[43]  P F Hall,et al.  Rethinking risk. , 1994, Canadian family physician Medecin de famille canadien.

[44]  Kae Nakamura,et al.  Coding of Task Reward Value in the Dorsal Raphe Nucleus , 2010, The Journal of Neuroscience.

[45]  Peter Dayan,et al.  Serotonin, Inhibition, and Negative Mood , 2007, PLoS Comput. Biol..

[46]  B. Balleine,et al.  Reward‐guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico‐basal ganglia networks , 2008, The European journal of neuroscience.

[47]  B. Balleine,et al.  Blockade of NMDA receptors in the dorsomedial striatum prevents action–outcome learning in instrumental conditioning , 2005, The European journal of neuroscience.

[48]  P. Corr,et al.  A two-dimensional neuropsychology of defense: fear/anxiety and defensive distance , 2004, Neuroscience & Biobehavioral Reviews.

[49]  N. Weinstein Unrealistic optimism about future life events , 1980 .

[50]  Luke Clark,et al.  Social and Emotional Decision-making Following Frontal Lobe Injury , 2004 .

[51]  M. G. King,et al.  Heart rate and muscle tension correlates of conditioned suppression in humans. , 1974, Journal of experimental psychology.

[52]  Q. Huys,et al.  Serotonin and Aversive Pavlovian Control of Instrumental Behavior in Humans , 2018 .

[53]  C. Spielberger,et al.  Manual for the State-Trait Anxiety Inventory , 1970 .

[54]  B. Moghaddam,et al.  Differential representation of Pavlovian–instrumental transfer by prefrontal cortex subregions and striatum , 2009, The European journal of neuroscience.

[55]  F. Graeff,et al.  Role of 5-HT in stress, anxiety, and depression , 1996, Pharmacology Biochemistry and Behavior.

[56]  P. Fletcher,et al.  Faculty Opinions recommendation of A selective role for dopamine in stimulus-reward learning. , 2011 .

[57]  Sham M. Kakade,et al.  Opponent interactions between serotonin and dopamine , 2002, Neural Networks.

[58]  M. Delgado,et al.  Avoidance‐based human Pavlovian‐to‐instrumental transfer , 2013, The European journal of neuroscience.

[59]  J. Wallis,et al.  Neuronal Mechanisms in Prefrontal Cortex Underlying Adaptive Choice Behavior , 2007, Annals of the New York Academy of Sciences.

[60]  N. Daw,et al.  Serotonin and Dopamine: Unifying Affective, Activational, and Decision Functions , 2011, Neuropsychopharmacology.

[61]  L. Claes,et al.  Gray's Reinforcement Sensitivity Theory as a framework for research on personality-psychopathology associations. , 2009, Clinical psychology review.

[62]  Antonio R. Damasio,et al.  Towards a neuropathology of emotion and mood , 1997, Nature.

[63]  C. Carter,et al.  Tryptophan Depletion Alters the Decision-Making of Healthy Volunteers through Altered Processing of Reward Cues , 2003, Neuropsychopharmacology.

[64]  Jakob Heinzle,et al.  Decoding different roles for vmPFC and dlPFC in multi-attribute decision making , 2011, NeuroImage.

[65]  Vivian V. Valentin,et al.  Determining the Neural Substrates of Goal-Directed Learning in the Human Brain , 2007, The Journal of Neuroscience.

[66]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[67]  B. Sahakian,et al.  Acute tryptophan depletion evokes negative mood in healthy females who have previously experienced concurrent negative mood and tryptophan depletion , 2009, Psychopharmacology.

[68]  T. Robbins,et al.  Converging evidence for central 5-HT effects in acute tryptophan depletion , 2012, Molecular Psychiatry.

[69]  A. Bond,et al.  The use of analogue scales in rating subjective feelings , 1974 .

[70]  Chandra Sekhar Sripada,et al.  Corticolimbic Function in Impulsive Aggressive Behavior , 2011, Biological Psychiatry.

[71]  S. Haber The primate basal ganglia: parallel and integrative networks , 2003, Journal of Chemical Neuroanatomy.

[72]  S. Haber,et al.  The Reward Circuit: Linking Primate Anatomy and Human Imaging , 2010, Neuropsychopharmacology.

[73]  T. Robbins,et al.  Serotoninergic regulation of emotional and behavioural control processes , 2008, Trends in Cognitive Sciences.

[74]  J. Price,et al.  The organization of networks within the orbital and medial prefrontal cortex of rats, monkeys and humans. , 2000, Cerebral cortex.

[75]  A. Beck,et al.  Comparison of Beck Depression Inventories -IA and -II in psychiatric outpatients. , 1996, Journal of personality assessment.

[76]  B. Sahakian,et al.  Recurrence in major depressive disorder: a neurocognitive perspective , 2007, Psychological Medicine.

[77]  B. Balleine,et al.  The General and Outcome-Specific Forms of Pavlovian-Instrumental Transfer Are Differentially Mediated by the Nucleus Accumbens Core and Shell , 2011, The Journal of Neuroscience.

[78]  Karl J. Friston,et al.  Modeling regional and psychophysiologic interactions in fMRI: the importance of hemodynamic deconvolution , 2003, NeuroImage.

[79]  Kae Nakamura,et al.  A Neural Correlate of Predicted and Actual Reward-Value Information in Monkey Pedunculopontine Tegmental and Dorsal Raphe Nucleus during Saccade Tasks , 2011, Neural plasticity.

[80]  J. Stevens,et al.  Animal Intelligence , 1883, Nature.

[81]  J. Patton,et al.  Factor structure of the Barratt impulsiveness scale. , 1995, Journal of clinical psychology.

[82]  B. Balleine,et al.  The role of the dorsomedial striatum in instrumental conditioning , 2005, The European journal of neuroscience.

[83]  T. Robinson,et al.  A selective role for dopamine in reward learning , 2010, Nature.

[84]  J. Deakin,et al.  5-HT and mechanisms of defence , 1991, Journal of psychopharmacology.

[85]  Nikolaus R. McFarland,et al.  Striatonigrostriatal Pathways in Primates Form an Ascending Spiral from the Shell to the Dorsolateral Striatum , 2000, The Journal of Neuroscience.

[86]  Kae Nakamura,et al.  Reward-Dependent Modulation of Neuronal Activity in the Primate Dorsal Raphe Nucleus , 2008, The Journal of Neuroscience.

[87]  D. Watson,et al.  Development and validation of brief measures of positive and negative affect: the PANAS scales. , 1988, Journal of personality and social psychology.

[88]  B. Balleine,et al.  The Neural Mechanisms Underlying the Influence of Pavlovian Cues on Human Decision Making , 2008, The Journal of Neuroscience.

[89]  Colin Camerer,et al.  Self-control in decision-making involves modulation of the vmPFC valuation system , 2009, NeuroImage.

[90]  C. Carver,et al.  Behavioral inhibition, behavioral activation, and affective responses to impending reward and punishment: The BIS/BAS Scales , 1994 .

[91]  S. Iversen,et al.  5-Hydroxytryptamine and punishment , 1977, Nature.

[92]  P. Dayan,et al.  Human Pavlovian–Instrumental Transfer , 2008, The Journal of Neuroscience.

[93]  Antonio Rangel,et al.  The Decision Value Computations in the vmPFC and Striatum Use a Relative Value Code That is Guided by Visual Attention , 2011, The Journal of Neuroscience.

[94]  P. Glimcher,et al.  The Neurobiology of Decision: Consensus and Controversy , 2009, Neuron.

[95]  P. Dayan,et al.  Serotonin Selectively Modulates Reward Value in Human Decision-Making , 2012, The Journal of Neuroscience.

[96]  A. van Dalen,et al.  Validation of the determination of amino acids in plasma by high-performance liquid chromatography using automated pre-column derivatization with o-phthaldialdehyde. , 1995, Journal of chromatography. B, Biomedical applications.

[97]  Karl J. Friston,et al.  Nonlinear Regression in Parametric Activation Studies , 1996, NeuroImage.

[98]  D. Norris,et al.  BOLD contrast sensitivity enhancement and artifact reduction with multiecho EPI: Parallel‐acquired inhomogeneity‐desensitized fMRI , 2006, Magnetic resonance in medicine.

[99]  Jonathan Evans Dual-processing accounts of reasoning, judgment, and social cognition. , 2008, Annual review of psychology.

[100]  Peter Dayan,et al.  Bonsai Trees in Your Head: How the Pavlovian System Sculpts Goal-Directed Choices by Pruning Decision Trees , 2012, PLoS Comput. Biol..

[101]  A. Dickinson,et al.  Differential Engagement of the Ventromedial Prefrontal Cortex by Goal-Directed and Habitual Behavior toward Food Pictures in Humans , 2009, The Journal of Neuroscience.

[102]  Raymond J. Dolan,et al.  Disentangling the Roles of Approach, Activation and Valence in Instrumental and Pavlovian Responding , 2011, PLoS Comput. Biol..

[103]  Daniel T. Knoepfle,et al.  Value Computations in Ventral Medial Prefrontal Cortex during Charitable Decision Making Incorporate Input from Regions Involved in Social Cognition , 2010, The Journal of Neuroscience.

[104]  B. Sahakian,et al.  Acute Tryptophan Depletion in Healthy Volunteers Enhances Punishment Prediction but Does not Affect Reward Prediction , 2008, Neuropsychopharmacology.

[105]  Karl J. Friston,et al.  Analysis of fMRI Time-Series Revisited—Again , 1995, NeuroImage.

[106]  M. Hamilton A RATING SCALE FOR DEPRESSION , 1960, Journal of neurology, neurosurgery, and psychiatry.

[107]  Colin Camerer,et al.  A framework for studying the neurobiology of value-based decision making , 2008, Nature Reviews Neuroscience.

[108]  K. Kirby,et al.  Delay-discounting probabilistic rewards: Rates decrease as amounts increase , 1996, Psychonomic bulletin & review.

[109]  P. Dayan,et al.  Serotonin in affective control. , 2009, Annual review of neuroscience.