Explicit and Implicit Reinforcement Learning Across the Psychosis Spectrum

Motivational and hedonic impairments are core features of a variety of types of psychopathology. An important aspect of motivational function is reinforcement learning (RL), including implicit (i.e., outside of conscious awareness) and explicit (i.e., including explicit representations about potential reward associations) learning, as well as both positive reinforcement (learning about actions that lead to reward) and punishment (learning to avoid actions that lead to loss). Here we present data from paradigms designed to assess both positive and negative components of both implicit and explicit RL, examine performance on each of these tasks among individuals with schizophrenia, schizoaffective disorder, and bipolar disorder with psychosis, and examine their relative relationships to specific symptom domains transdiagnostically. None of the diagnostic groups differed significantly from controls on the implicit RL tasks in either bias toward a rewarded response or bias away from a punished response. However, on the explicit RL task, both the individuals with schizophrenia and schizoaffective disorder performed significantly worse than controls, but the individuals with bipolar did not. Worse performance on the explicit RL task, but not the implicit RL task, was related to worse motivation and pleasure symptoms across all diagnostic categories. Performance on explicit RL, but not implicit RL, was related to working memory, which accounted for some of the diagnostic group differences. However, working memory did not account for the relationship of explicit RL to motivation and pleasure symptoms. These findings suggest transdiagnostic relationships across the spectrum of psychotic disorders between motivation and pleasure impairments and explicit RL.

[1]  Jr. William Rush Dunton,et al.  THE AMERICAN JOURNAL OF PSYCHIATRY , 1944 .

[2]  D. A. Pond,et al.  Social Class and Mental Illness , 1958, Mental Health.

[3]  Fredrick C. Redlich,et al.  Social class and mental illness: Community study. , 1958 .

[4]  J. Overall,et al.  The Brief Psychiatric Rating Scale , 1962 .

[5]  R. C. Young,et al.  A Rating Scale for Mania: Reliability, Validity and Sensitivity , 1978, British Journal of Psychiatry.

[6]  E. Struening,et al.  SLOF: a behavioral rating scale for assessing the mentally ill. , 1983, Social work research & abstracts.

[7]  K. Mueser,et al.  Social learning for chronic mental inpatients. , 1986, Schizophrenia bulletin.

[8]  A. Leach,et al.  Negative symptoms , 1991 .

[9]  Xiao-Li Meng,et al.  Comparing correlated correlation coefficients , 1992 .

[10]  W. Schultz Activity of dopamine neurons in the behaving primate , 1992 .

[11]  R. Liberman,et al.  Brief Psychiatric Rating Scale (BPRS) Expanded Version (4.0): Scales, Anchor Points, and Administration Manual , 1993 .

[12]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[13]  Robert Paul Liberman,et al.  Training and quality assurance with the Brief Psychiatric Rating Scale: "The drift busters." , 1993 .

[14]  T. Robbins,et al.  Neuropsychological evidence for frontostriatal dysfunction in schizophrenia , 1995, Psychological Medicine.

[15]  R. Oades Stimulus dimension shifts in patients with schizophrenia, with and without paranoid hallucinatory symptoms, or obsessive compulsive disorder: strategies, blocking and monoamine status , 1997, Behavioural Brain Research.

[16]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[17]  T. Robbins,et al.  Executive function in first-episode schizophrenia , 1998, Psychological Medicine.

[18]  T. Robbins,et al.  Comparison of set-shifting ability in patients with chronic schizophrenia and frontal lobe damage , 1999, Schizophrenia Research.

[19]  Brian Knutson,et al.  FMRI Visualization of Brain Activity during a Monetary Incentive Delay Task , 2000, NeuroImage.

[20]  A. Antal,et al.  Schizophrenics know more than they can tell: probabilistic classification learning in schizophrenia , 2000, Psychological Medicine.

[21]  Michael J. Frank,et al.  Interactions between frontal cortex and basal ganglia in working memory: A computational model , 2001, Cognitive, affective & behavioral neuroscience.

[22]  E. Miller,et al.  An integrative theory of prefrontal cortex function. , 2001, Annual review of neuroscience.

[23]  T L Patterson,et al.  UCSD Performance-Based Skills Assessment , 2016 .

[24]  Brian Knutson,et al.  Dissociation of reward anticipation and outcome with event-related fMRI , 2001, Neuroreport.

[25]  P. Dayan,et al.  Reward, Motivation, and Reinforcement Learning , 2002, Neuron.

[26]  J. Malley,et al.  Habit and skill learning in schizophrenia: evidence of normal striatal processing with abnormal cortical input. , 2002, Learning & memory.

[27]  T. Patterson,et al.  Generalized cognitive impairments, ability to perform everyday tasks, and level of independence in community living situations of older patients with psychosis. , 2002, The American journal of psychiatry.

[28]  T. Robbins,et al.  Executive dysfunction in first-episode schizophrenia and relationship to duration of untreated psychosis: The West London Study , 2002, British Journal of Psychiatry.

[29]  Jennifer A. Mangels,et al.  Typical and atypical antipsychotic medications differentially affect two nondeclarative memory tasks in schizophrenic patients: a double dissociation , 2003, Schizophrenia Research.

[30]  Samuel M. McClure,et al.  Temporal Prediction Errors in a Passive Learning Task Activate Human Striatum , 2003, Neuron.

[31]  A. Mortimer,et al.  Stability of set-shifting and planning abilities in patients with schizophrenia , 2004, Psychiatry Research.

[32]  K. Berridge Motivation concepts in behavioral neuroscience , 2004, Physiology & Behavior.

[33]  W. Schultz Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology , 2004, Current Opinion in Neurobiology.

[34]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[35]  Luke Clark,et al.  Modafinil Improves Cognition and Attentional Set Shifting in Patients with Chronic Schizophrenia , 2004, Neuropsychopharmacology.

[36]  Martin P Paulus,et al.  Increased sensitivity to error during decision-making in bipolar disorder patients with acute mania. , 2004, Journal of affective disorders.

[37]  E. Leibenluft,et al.  Reward-related processes in pediatric bipolar disorder: a pilot study. , 2004, Journal of affective disorders.

[38]  E. Leibenluft,et al.  The Impact of Reward, Punishment, and Frustration on Attention in Pediatric Bipolar Disorder , 2005, Biological Psychiatry.

[39]  Amanda Price Cortico-striatal contributions to category learning: dissociating the verbal and implicit systems. , 2005, Behavioral neuroscience.

[40]  Daniel S Pine,et al.  Deficits on a probabilistic response-reversal task in patients with pediatric bipolar disorder. , 2005, The American journal of psychiatry.

[41]  D. Pizzagalli,et al.  Toward an objective characterization of an anhedonic phenotype: A signal-detection approach , 2005, Biological Psychiatry.

[42]  T. Robbins,et al.  Serotonergic Modulation of Prefrontal Cortex during Negative Feedback in Probabilistic Reversal Learning , 2005, Neuropsychopharmacology.

[43]  Catherine E. Myers,et al.  Dissociation between medial temporal lobe and basal ganglia memory systems in schizophrenia , 2005, Schizophrenia Research.

[44]  D. Pizzagalli,et al.  Acute Stress Reduces Reward Responsiveness: Implications for Depression , 2006, Biological Psychiatry.

[45]  J. O'Doherty,et al.  Is Avoiding an Aversive Outcome Rewarding? Neural Substrates of Avoidance Learning in the Human Brain , 2006, PLoS biology.

[46]  Michael J. Frank,et al.  A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. , 2006, Behavioral neuroscience.

[47]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[48]  Thomas E. Smith,et al.  Behavioral Rehabilitation of the "Treatment-Refractory" Schizophrenia Patient: Conceptual Foundations, Interventions, and Outcome Data , 2006 .

[49]  Henrik Walter,et al.  Prediction error as a linear function of reward probability is coded in human nucleus accumbens , 2006, NeuroImage.

[50]  James A. Waltz,et al.  Probabilistic reversal learning impairments in schizophrenia: Further evidence of orbitofrontal dysfunction , 2007, Schizophrenia Research.

[51]  W. Schultz Multiple dopamine functions at different time courses. , 2007, Annual review of neuroscience.

[52]  Sandra Jazbec,et al.  Intra-dimensional/extra-dimensional set-shifting performance in schizophrenia: Impact of distractors , 2007, Schizophrenia Research.

[53]  Philip D. Harvey,et al.  Performance-based measures of functional skills: usefulness in clinical treatment studies. , 2007, Schizophrenia bulletin.

[54]  Thomas E. Hazy,et al.  Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[55]  D. Lagnado,et al.  Challenging the role of implicit processes in probabilistic category learning , 2007, Psychonomic bulletin & review.

[56]  Benjamin M. Robinson,et al.  Selective Reinforcement Learning Deficits in Schizophrenia Support Predictions from Computational Models of Striatal-Cortical Dysfunction , 2007, Biological Psychiatry.

[57]  M. Tohen,et al.  The Bipolar Depression Rating Scale (BDRS): its development, validation and utility. , 2007, Bipolar disorders.

[58]  E. Harmon-Jones,et al.  Effect of Bipolar Disorder on Left Frontal Cortical Responses to Goals Differing in Valence and Task Difficulty , 2008, Biological Psychiatry.

[59]  Catherine E. Myers,et al.  How to find the way out from four rooms? The learning of “chaining” associations may shed light on the neuropsychology of the deficit syndrome of schizophrenia , 2008, Schizophrenia Research.

[60]  Catherine E. Myers,et al.  Associative learning in deficit and nondeficit schizophrenia , 2008, Neuroreport.

[61]  M. Fava,et al.  Reduced hedonic capacity in major depressive disorder: evidence from a probabilistic reward task. , 2008, Journal of psychiatric research.

[62]  M. Egan,et al.  Set-Shifting Ability and Schizophrenia: A Marker of Clinical Illness or an Intermediate Phenotype? , 2008, Biological Psychiatry.

[63]  E. Leibenluft,et al.  Processing of differentially valued rewards and punishments in youths with bipolar disorder or severe mood dysregulation. , 2008, Journal of child and adolescent psychopharmacology.

[64]  J. Gold,et al.  Decision-Making Impairments in the Context of Intact Reward Sensitivity in Schizophrenia , 2008, Biological Psychiatry.

[65]  Clay B. Holroyd,et al.  Learning-related changes in brain activity following errors and performance feedback in schizophrenia , 2008, Schizophrenia Research.

[66]  Elena Goetz,et al.  Euthymic Patients with Bipolar Disorder Show Decreased Reward Learning in a Probabilistic Reward Task , 2008, Biological Psychiatry.

[67]  A M McIntosh,et al.  Set shifting and reversal learning in patients with bipolar disorder or schizophrenia , 2008, Psychological Medicine.

[68]  E. T. Bullmore,et al.  Reinforcement and Reversal Learning in First-Episode Psychosis , 2008, Schizophrenia bulletin.

[69]  J. Nurnberger,et al.  A multimethod investigation of the behavioral activation system in bipolar disorder. , 2008, Journal of abnormal psychology.

[70]  B. Abler,et al.  Altered reward functions in patients on atypical antipsychotic medication in line with the revised dopamine hypothesis of schizophrenia , 2009, Psychopharmacology.

[71]  E. Leibenluft,et al.  Impaired probabilistic reversal learning in youths with mood and anxiety disorders , 2009, Psychological Medicine.

[72]  S. Shamay-Tsoory,et al.  Neuropsychological Evidence of Impaired Cognitive Empathy in Euthymic Bipolar Disorder , 2009 .

[73]  G. Juckel,et al.  Impairment of probabilistic reward-based learning in schizophrenia. , 2009, Neuropsychology.

[74]  M. Frank,et al.  Genetic contributions to avoidance-based decisions: striatal D2 receptor polymorphisms , 2009, Neuroscience.

[75]  S. Shamay-Tsoory,et al.  Neuropsychological evidence of impaired cognitive empathy in euthymic bipolar disorder. , 2009, The Journal of neuropsychiatry and clinical neurosciences.

[76]  T. Robbins,et al.  Reconciling the Role of Serotonin in Behavioral Inhibition and Aversion: Acute Tryptophan Depletion Abolishes Punishment-Induced Inhibition in Humans , 2009, The Journal of Neuroscience.

[77]  Catherine E. Myers,et al.  Relative Risk of Probabilistic Category Learning Deficits in Patients with Schizophrenia and Their Siblings , 2010, Biological Psychiatry.

[78]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[79]  T. Robbins,et al.  Serotonin Modulates Sensitivity to Reward and Negative Feedback in a Probabilistic Reversal Learning Task in Rats , 2010, Neuropsychopharmacology.

[80]  R. Baldessarini,et al.  International consensus study of antipsychotic dosing. , 2010, The American journal of psychiatry.

[81]  T. Insel,et al.  Wesleyan University From the SelectedWorks of Charles A . Sanislow , Ph . D . 2010 Research Domain Criteria ( RDoC ) : Toward a New Classification Framework for Research on Mental Disorders , 2018 .

[82]  Gerd Wagner,et al.  Altered activation in association with reward-related trial-and-error learning in patients with schizophrenia , 2010, NeuroImage.

[83]  S. Strakowski,et al.  Impulsivity across the course of bipolar disorder. , 2010, Bipolar disorders.

[84]  Michael G. Hardin,et al.  Perturbed reward processing in pediatric bipolar disorder: an antisaccade study , 2010, Journal of psychopharmacology.

[85]  E. Leibenluft,et al.  Altered neural function in pediatric bipolar disorder during reversal learning. , 2010, Bipolar disorders.

[86]  Catherine E. Myers,et al.  General functioning predicts reward and punishment learning in schizophrenia , 2011, Schizophrenia Research.

[87]  Thomas V. Wiecki,et al.  Altered probabilistic learning and response biases in schizophrenia: behavioral evidence and neurocomputational modeling. , 2011, Neuropsychology.

[88]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[89]  M. Frank,et al.  Deficits in Positive Reinforcement Learning and Uncertainty-Driven Exploration Are Associated with Distinct Aspects of Negative Symptoms in Schizophrenia , 2011, Biological Psychiatry.

[90]  Michael J Frank,et al.  Negative symptoms and the failure to represent the expected reward value of actions: behavioral and computational modeling evidence. , 2012, Archives of general psychiatry.

[91]  J. Levitt,et al.  The relationship between reward-based learning and nicotine dependence in smokers with schizophrenia , 2012, Psychiatry Research.

[92]  Deanna M Barch,et al.  Optimization of a goal maintenance task for use in clinical applications. , 2012, Schizophrenia bulletin.

[93]  J. Ragland,et al.  Relational and Item-Specific Encoding (RISE): task development and psychometric characteristics. , 2012, Schizophrenia bulletin.

[94]  J. Ragland,et al.  Optimization and validation of a visual integration test for schizophrenia research. , 2012, Schizophrenia bulletin.

[95]  S. Luck,et al.  The clinical translation of a measure of gain control: the contrast-contrast effect task. , 2012, Schizophrenia bulletin.

[96]  Michael D. Edge,et al.  The behavioral activation system and mania. , 2012, Annual review of clinical psychology.

[97]  N. Daw,et al.  The ubiquity of model-based reinforcement learning , 2012, Current Opinion in Neurobiology.

[98]  T. Carmody,et al.  Cognitive endophenotypes of psychosis within dimension and diagnosis , 2012, Psychiatry Research.

[99]  T. Braver The variable nature of cognitive control: a dual mechanisms framework , 2012, Trends in Cognitive Sciences.

[100]  Joey Tang,et al.  Schizophrenia and risk-taking: Impaired reward but preserved punishment processing , 2012, Schizophrenia Research.

[101]  A. Gonul,et al.  Reduced reward-related probability learning in schizophrenia patients , 2012, Neuropsychiatric disease and treatment.

[102]  S. Luck,et al.  Clinical, functional, and intertask correlations of measures developed by the Cognitive Neuroscience Test Reliability and Clinical Applications for Schizophrenia Consortium. , 2012, Schizophrenia bulletin.

[103]  Ágnes Lukács,et al.  Self-Insight in Probabilistic Category Learning , 2013, The Journal of general psychology.

[104]  J. Hodges,et al.  Probabilistic association learning in frontotemporal dementia and schizophrenia , 2013, Cortex.

[105]  R. Gur,et al.  The Clinical Assessment Interview for Negative Symptoms (CAINS): final development and validation. , 2013, The American journal of psychiatry.

[106]  R. McMahon,et al.  The Roles of Reward, Default, and Executive Control Networks in Set-Shifting Impairments in Schizophrenia , 2013, PloS one.

[107]  C. S. Carter,et al.  Cognition in schizophrenia and schizo-affective disorder: impairments that are more similar than different , 2013, Psychological Medicine.

[108]  G. Remington,et al.  Impairments in both reward and punishment guided reinforcement learning in schizophrenia , 2013, Schizophrenia Research.

[109]  John A Sweeney,et al.  Generalized and specific neurocognitive deficits in psychotic disorders: utility for evaluating pharmacological treatment effects and as intermediate phenotypes for gene discovery. , 2014, Schizophrenia bulletin.

[110]  Godfrey Pearlson,et al.  Bipolar and schizophrenia network for intermediate phenotypes: outcomes across the psychosis continuum. , 2014, Schizophrenia bulletin.

[111]  Ferenc Kemény,et al.  Self-insight in probabilistic categorization – not implicit in children either , 2014, Front. Psychol..

[112]  Anne G E Collins,et al.  Working Memory Contributions to Reinforcement Learning Impairments in Schizophrenia , 2014, The Journal of Neuroscience.

[113]  S. Kapur,et al.  Impaired subjective well-being in schizophrenia is associated with reduced anterior cingulate activity during reward processing , 2014, Psychological Medicine.

[114]  Edward E. Smith,et al.  Patients with schizophrenia are impaired when learning in the context of pursuing rewards , 2014, Schizophrenia Research.

[115]  D. Barch,et al.  The motivation and pleasure dimension of negative symptoms: Neural substrates and behavioral outputs , 2014, European Neuropsychopharmacology.

[116]  Shinsuke Shimojo,et al.  Neural Computations Underlying Arbitration between Model-Based and Model-free Learning , 2013, Neuron.

[117]  J. Kerns,et al.  Reinforcement learning deficits in people with schizophrenia persist after extended trials , 2014, Psychiatry Research.

[118]  T. Insel The NIMH Research Domain Criteria (RDoC) Project: precision medicine for psychiatry. , 2014, The American journal of psychiatry.

[119]  Katherine R. Luking,et al.  Child Gain Approach and Loss Avoidance Behavior: Relationships With Depression Risk, Negative Mood, and Anhedonia. , 2015, Journal of the American Academy of Child and Adolescent Psychiatry.

[120]  R. Nusslock,et al.  The development and course of bipolar spectrum disorders: an integrated reward and circadian rhythm dysregulation model. , 2015, Annual review of clinical psychology.

[121]  N. Daw,et al.  Cognitive Control Predicts Use of Model-based Reinforcement Learning , 2014, Journal of Cognitive Neuroscience.

[122]  William A. Cunningham,et al.  Elevated striatal reactivity across monetary and social rewards in bipolar I disorder. , 2015, Journal of abnormal psychology.

[123]  Roshan Cools,et al.  Impaired Activation in Cognitive Control Regions Predicts Reversal Learning in Schizophrenia. , 2016, Schizophrenia bulletin.

[124]  Shannon J Drayton,et al.  Bipolar depression , 2016, International journal of psychiatry in medicine.

[125]  A. Hill,et al.  Wechsler Test of Adult Reading , 2017 .

[126]  Katherine R. Luking,et al.  Reduced Hedonic Capacity/Approach Motivation Relates to Blunted Responsivity to Gain and Loss Feedback in Children , 2017, Journal of clinical child and adolescent psychology : the official journal for the Society of Clinical Child and Adolescent Psychology, American Psychological Association, Division 53.