Motivational deficits in schizophrenia relate to abnormalities in cortical learning rate signals

Individuals from across the psychosis spectrum display impairments in reinforcement learning. In some individuals, these deficits may result from aberrations in reward prediction error (RPE) signaling, conveyed by dopaminergic projections to the ventral striatum (VS). However, there is mounting evidence that VS RPE signals are relatively intact in medicated people with schizophrenia (PSZ). We hypothesized that, in PSZ, reinforcement learning deficits often are not related to RPE signaling per se but rather their impact on learning and behavior (i.e., learning rate modulation), due to dysfunction in anterior cingulate and dorsomedial prefrontal cortex (dmPFC). Twenty-six PSZ and 23 healthy volunteers completed a probabilistic reinforcement learning paradigm with occasional, sudden, shifts in contingencies. Using computational modeling, we found evidence of an impairment in trial-wise learning rate modulation (α) in PSZ before and after a reinforcement contingency shift, expressed most in PSZ with more severe motivational deficits. In a subsample of 22 PSZ and 22 healthy volunteers, we found little evidence for between-group differences in VS RPE and dmPFC learning rate signals, as measured with fMRI. However, a follow-up psychophysiological interaction analysis revealed decreased dmPFC-VS connectivity concurrent with learning rate modulation, most prominently in individuals with the most severe motivational deficits. These findings point to an impairment in learning rate modulation in PSZ, leading to a reduced ability to adjust task behavior in response to unexpected outcomes. At the level of the brain, learning rate modulation deficits may be associated with decreased involvement of the dmPFC within a greater RL network.

[1]  C. Pantelis,et al.  Altered corticostriatal functional connectivity in individuals with high social anhedonia , 2015, Psychological Medicine.

[2]  Katharine N. Thakkar,et al.  Reduced error-related activation in two anterior cingulate circuits is related to impaired performance in schizophrenia. , 2008, Brain : a journal of neurology.

[3]  Michael F. Green,et al.  Probabilistic Reversal Learning in Schizophrenia: Stability of Deficits and Potential Causal Mechanisms. , 2016, Schizophrenia bulletin.

[4]  J. Kerns,et al.  Reinforcement learning deficits in people with schizophrenia persist after extended trials , 2014, Psychiatry Research.

[5]  E T Bullmore,et al.  Substantia nigra/ventral tegmental reward prediction error disruption in psychosis , 2008, Molecular Psychiatry.

[6]  Jonathan D. Cohen,et al.  Decreased conflict- and error-related activity in the anterior cingulate cortex in subjects with schizophrenia. , 2005, The American journal of psychiatry.

[7]  J. Waltz,et al.  Motivational Deficits in Schizophrenia and the Representation of Expected Value. , 2016, Current topics in behavioral neurosciences.

[8]  G. Waiter,et al.  Expected value and prediction error abnormalities in depression and schizophrenia. , 2011, Brain : a journal of neurology.

[9]  Timothy E. J. Behrens,et al.  Tools of the trade: psychophysiological interactions and functional connectivity. , 2012, Social cognitive and affective neuroscience.

[10]  Jared X. Van Snellenberg,et al.  Motivational Context Modulates Prediction Error Response in Schizophrenia. , 2016, Schizophrenia bulletin.

[11]  Josiah R. Boivin,et al.  A Causal Link Between Prediction Errors, Dopamine Neurons and Learning , 2013, Nature Neuroscience.

[12]  Raymond J. Dolan,et al.  Striatal dysfunction during reversal learning in unmedicated schizophrenia patients☆ , 2014, NeuroImage.

[13]  W. Schultz,et al.  Dopamine Modulates Adaptive Prediction Error Coding in the Human Midbrain and Striatum , 2017, The Journal of Neuroscience.

[14]  Michael J Frank,et al.  A cholinergic feedback circuit to regulate striatal population uncertainty and optimize reinforcement learning , 2015, eLife.

[15]  M. Frank,et al.  An Integrative Perspective on the Role of Dopamine in Schizophrenia , 2017, Biological Psychiatry.

[16]  Robert C. Wilson,et al.  Is Model Fitting Necessary for Model-Based fMRI? , 2015, PLoS Comput. Biol..

[17]  Deanna M Barch,et al.  Probabilistic Reinforcement Learning in Patients With Schizophrenia: Relationships to Anhedonia and Avolition. , 2016, Biological psychiatry. Cognitive neuroscience and neuroimaging.

[18]  E T Bullmore,et al.  How dopamine dysregulation leads to psychotic symptoms? Abnormal mesolimbic and mesostriatal prediction error signalling in psychosis , 2008, Molecular Psychiatry.

[19]  P. McGuire,et al.  Ventral Striatal Activation During Reward Processing in Psychosis: A Neurofunctional Meta-Analysis. , 2015, JAMA psychiatry.

[20]  A. Malhotra,et al.  Antipsychotic treatment and functional connectivity of the striatum in first-episode schizophrenia. , 2015, JAMA psychiatry.

[21]  Sheri L. Johnson,et al.  Explicit and Implicit Reinforcement Learning Across the Psychosis Spectrum , 2017, Journal of abnormal psychology.

[22]  D. Weinberger,et al.  Estimating changing contexts in schizophrenia. , 2016, Brain : a journal of neurology.

[23]  Peter N. C. Mohr,et al.  Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions , 2009, Proceedings of the National Academy of Sciences.

[24]  Anne G E Collins,et al.  Working Memory Contributions to Reinforcement Learning Impairments in Schizophrenia , 2014, The Journal of Neuroscience.

[25]  Gerd Wagner,et al.  Altered activation in association with reward-related trial-and-error learning in patients with schizophrenia , 2010, NeuroImage.

[26]  R. Conley,et al.  Novel Factor-Based Symptom Scores in Treatment Resistant Schizophrenia: Implications for Clinical Trials , 2002, Neuropsychopharmacology.

[27]  Ivan Toni,et al.  Neural dynamics of error processing in medial frontal cortex , 2005, NeuroImage.

[28]  Michael F. Green,et al.  The MATRICS Consensus Cognitive Battery, part 1: test selection, reliability, and validity. , 2008, The American journal of psychiatry.

[29]  G. Strauss,et al.  A Review of Anticipatory Pleasure in Schizophrenia , 2016, Current Behavioral Neuroscience Reports.

[30]  Martin P Paulus,et al.  Schizophrenia Subjects Show Intact Success-Related Neural Activation but Impaired Uncertainty Processing during Decision-Making , 2003, Neuropsychopharmacology.

[31]  Dennis Hernaus,et al.  Noradrenaline transporter blockade increases fronto-parietal functional connectivity relevant for working memory , 2017, European Neuropsychopharmacology.

[32]  A. Fritzsche,et al.  Anticipatory pleasure and approach motivation in schizophrenia-like negative symptoms , 2013, Psychiatry Research.

[33]  Alexander L. Miller,et al.  Executive-frontal lobe cognitive dysfunction in schizophrenia: A symptom subtype analysis , 1998, Psychiatry Research.

[34]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[35]  Russell A. Poldrack,et al.  Dynamic fluctuations in global brain network topology characterize functional states during rest and behavior , 2015, 1511.02976.

[36]  M. First,et al.  Structured clinical interview for DSM-IV axis I disorders : SCID-I : clinical version : scoresheet , 1997 .

[37]  James A. Waltz,et al.  Probabilistic reversal learning impairments in schizophrenia: Further evidence of orbitofrontal dysfunction , 2007, Schizophrenia Research.

[38]  R W Cox,et al.  AFNI: software for analysis and visualization of functional magnetic resonance neuroimages. , 1996, Computers and biomedical research, an international journal.

[39]  David Badre,et al.  Working Memory Load Strengthens Reward Prediction Errors , 2017, The Journal of Neuroscience.

[40]  Matthew R. Nassar,et al.  Catecholaminergic Regulation of Learning Rate in a Dynamic Environment , 2016, PLoS Comput. Biol..

[41]  Krzysztof J. Gorgolewski,et al.  The Dynamics of Functional Brain Networks: Integrated Network States during Cognitive Task Performance , 2015, Neuron.

[42]  D. Wechsler Wechsler Abbreviated Scale of Intelligence--Second Edition , 2018 .

[43]  N. Andreasen,et al.  Antipsychotic Dose Equivalents and Dose-Years: A Standardized Method for Comparing Exposure to Different Drugs , 2010, Biological Psychiatry.

[44]  S. Shergill,et al.  Dysfunctional Striatal Systems in Treatment-Resistant Schizophrenia , 2016, Neuropsychopharmacology.

[45]  R. McMahon,et al.  The Roles of Reward, Default, and Executive Control Networks in Set-Shifting Impairments in Schizophrenia , 2013, PloS one.

[46]  S. Kapur,et al.  Improvement of brain reward abnormalities by antipsychotic monotherapy in schizophrenia. , 2012, Archives of general psychiatry.

[47]  M. Geyer,et al.  Long-range correlations in choice sequences of schizophrenic patients , 1999, Schizophrenia Research.

[48]  J. Waltz,et al.  Probing the Dynamic Updating of Value in Schizophrenia Using a Sensory-Specific Satiety Paradigm. , 2015, Schizophrenia bulletin.

[49]  P. Tobler,et al.  Deficits in reinforcement learning but no link to apathy in patients with schizophrenia , 2017, Scientific Reports.

[50]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[51]  R. Buchert,et al.  Aberrant Salience Is Related to Reduced Reinforcement Learning Signals and Elevated Dopamine Synthesis Capacity in Healthy Adults , 2015, The Journal of Neuroscience.

[52]  Michael J Frank,et al.  Negative symptoms and the failure to represent the expected reward value of actions: behavioral and computational modeling evidence. , 2012, Archives of general psychiatry.

[53]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[54]  P. Dayan,et al.  Model-based and model-free Pavlovian reward learning: Revaluation, revision, and revelation , 2014, Cognitive, affective & behavioral neuroscience.

[55]  Joseph T. McGuire,et al.  Functionally Dissociable Influences on Learning Rate in a Dynamic Environment , 2014, Neuron.

[56]  B. Franke,et al.  Dissociable Effects of Dopamine and Serotonin on Reversal Learning , 2013, Neuron.

[57]  J. Dreher,et al.  The medial orbitofrontal cortex encodes a general unsigned value signal during anticipation of both appetitive and aversive events , 2015, Cortex.

[58]  Elizabeth Thompson,et al.  Reinforcement Learning Performance and Risk for Psychosis in Youth , 2015, The Journal of nervous and mental disease.

[59]  J. Waltz,et al.  A review of reward processing and motivational impairment in schizophrenia. , 2014, Schizophrenia bulletin.

[60]  J. Schweitzer,et al.  Abnormal Responses to Monetary Outcomes in Cortex, but not in the Basal Ganglia, in Schizophrenia , 2010, Neuropsychopharmacology.

[61]  M. Bell,et al.  Neurocognitive function and insight in schizophrenia: support for an association with impairments in executive function but not with impairments in global function , 1998, Acta psychiatrica Scandinavica.

[62]  D. Barch,et al.  Intact Ventral Striatal Prediction Error Signaling in Medicated Schizophrenia Patients. , 2016, Biological psychiatry. Cognitive neuroscience and neuroimaging.