Altered probabilistic learning and response biases in schizophrenia: behavioral evidence and neurocomputational modeling.

OBJECTIVE Patients with schizophrenia (SZ) show reinforcement learning impairments related to both the gradual/procedural acquisition of reward contingencies, and the ability to use trial-to-trial feedback to make rapid behavioral adjustments. METHOD We used neurocomputational modeling to develop plausible mechanistic hypotheses explaining reinforcement learning impairments in individuals with SZ. We tested the model with a novel Go/NoGo learning task in which subjects had to learn to respond or withhold responses when presented with different stimuli associated with different probabilities of gains or losses in points. We analyzed data from 34 patients and 23 matched controls, characterizing positive- and negative-feedback-driven learning in both a training phase and a test phase. RESULTS Consistent with simulations from a computational model of aberrant dopamine input to the basal ganglia patients, patients with SZ showed an overall increased rate of responding in the training phase, together with reduced response-time acceleration to frequently rewarded stimuli across training blocks, and a reduced relative preference for frequently rewarded training stimuli in the test phase. Patients did not differ from controls on measures of procedural negative-feedback-driven learning, although patients with SZ exhibited deficits in trial-to-trial adjustments to negative feedback, with these measures correlating with negative symptom severity. CONCLUSIONS These findings support the hypothesis that patients with SZ have a deficit in procedural "Go" learning, linked to abnormalities in DA transmission at D1-type receptors, despite a "Go bias" (increased response rate), potentially related to excessive tonic dopamine. Deficits in trial-to-trial reinforcement learning were limited to a subset of patients with SZ with severe negative symptoms, putatively stemming from prefrontal cortical dysfunction.

[1]  P. Llorca,et al.  Regional cerebral blood flow in deficit/nondeficit types of schizophrenia according to SDS criteria , 2002, Progress in Neuro-Psychopharmacology and Biological Psychiatry.

[2]  J. Joyce,et al.  D2 but not D3 receptors are elevated after 9 or 11 months chronic haloperidol treatment: Influence of withdrawal period , 2001, Synapse.

[3]  H. Moore,et al.  Prefrontal DA Transmission at D1 Receptors and the Pathology of Schizophrenia , 2003, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[4]  James A. Waltz,et al.  Probabilistic reversal learning impairments in schizophrenia: Further evidence of orbitofrontal dysfunction , 2007, Schizophrenia Research.

[5]  Stefano Pini,et al.  Patterns of structural MRI abnormalities in deficit and nondeficit schizophrenia. , 2007, Schizophrenia bulletin.

[6]  M. Roesch,et al.  Orbitofrontal Cortex, Associative Learning, and Expectancies , 2005, Neuron.

[7]  A. Graybiel Habits, rituals, and the evaluative brain. , 2008, Annual review of neuroscience.

[8]  B. Kirkpatrick The Neural Basis of the Deficit Syndrome of Schizophrenia , 1990, The Journal of nervous and mental disease.

[9]  Niels G. Waller The Scale for the Assessment of Negative Symptoms , 1995 .

[10]  N. Andreasen The Scale for the Assessment of Negative Symptoms (SANS): Conceptual and Theoretical Foundations , 1989, British Journal of Psychiatry.

[11]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[12]  Philip Seeman,et al.  Dopamine supersensitivity correlates with D2High states, implying many paths to psychosis. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Benjamin M. Robinson,et al.  Selective Reinforcement Learning Deficits in Schizophrenia Support Predictions from Computational Models of Striatal-Cortical Dysfunction , 2007, Biological Psychiatry.

[14]  P. Calabresi,et al.  Chronic Haloperidol Promotes Corticostriatal Long-Term Potentiation by Targeting Dopamine D2L Receptors , 2004, The Journal of Neuroscience.

[15]  M. Frank,et al.  Prefrontal and striatal dopaminergic genes predict individual differences in exploration and exploitation. , 2009, Nature neuroscience.

[16]  Joshua L Plotkin,et al.  Differential Excitability and Modulation of Striatal Medium Spiny Neuron Dendrites , 2008, The Journal of Neuroscience.

[17]  A. Meyer-Lindenberg,et al.  Reduced prefrontal activity predicts exaggerated striatal dopaminergic function in schizophrenia , 2002, Nature Neuroscience.

[18]  J. Overall,et al.  The Brief Psychiatric Rating Scale , 1962 .

[19]  B. Balleine,et al.  A specific role for posterior dorsolateral striatum in human habit learning , 2009, The European journal of neuroscience.

[20]  F. Scorza,et al.  Progress in neuro-psychopharmacology and biological psychiatry Re.: Omega-3 fatty acids and sudden unexpected death in epilepsy: What does the evidence tell us? , 2007, Progress in Neuro-Psychopharmacology and Biological Psychiatry.

[21]  Ralph H. B. Benedict,et al.  Hopkins Verbal Learning Test--Revised , 2013 .

[22]  Shiaoching Gong,et al.  A gene expression atlas of the central nervous system based on bacterial artificial chromosomes , 2003, Nature.

[23]  M. Gluck,et al.  Reward-learning and the novelty-seeking personality: a between- and within-subjects study of the effects of dopamine agonists on young Parkinson's patients. , 2009, Brain : a journal of neurology.

[24]  T. Goldberg,et al.  Auditory working memory and Wisconsin Card Sorting Test performance in schizophrenia. , 1997, Archives of general psychiatry.

[25]  Petra E. Pajtas,et al.  Single dose of a dopamine agonist impairs reinforcement learning in humans: Behavioral evidence from a laboratory-based measure of reward responsiveness , 2008, Psychopharmacology.

[26]  B. Christensen,et al.  Measuring premorbid IQ in traumatic brain injury: An examination of the validity of the Wechsler Test of Adult Reading (WTAR) , 2008, Journal of clinical and experimental neuropsychology.

[27]  S. Galderisi,et al.  Conditional associative learning in drug-free schizophrenic patients. , 1987, Neuropsychobiology.

[28]  R. Murray,et al.  Episodic memory and learning in patients with chronic schizophrenia , 1999, Schizophrenia Research.

[29]  P. Greengard,et al.  Dichotomous Dopaminergic Control of Striatal Synaptic Plasticity , 2008, Science.

[30]  Samuel M. McClure,et al.  Patients with schizophrenia have a reduced neural response to both unpredictable and predictable primary reinforcers , 2008, Neuropsychopharmacology.

[31]  J. Malley,et al.  Habit and skill learning in schizophrenia: evidence of normal striatal processing with abnormal cortical input. , 2002, Learning & memory.

[32]  R. Malenka,et al.  Impaired central error-correcting behavior in schizophrenia. , 1982, Archives of general psychiatry.

[33]  Gerd Wagner,et al.  Altered activation in association with reward-related trial-and-error learning in patients with schizophrenia , 2010, NeuroImage.

[34]  R. Conley,et al.  Novel Factor-Based Symptom Scores in Treatment Resistant Schizophrenia: Implications for Clinical Trials , 2002, Neuropsychopharmacology.

[35]  Michael J. Frank,et al.  A mechanistic account of striatal dopamine function in human cognition: psychopharmacological studies with cabergoline and haloperidol. , 2006, Behavioral neuroscience.

[36]  A. Malhotra,et al.  Why Not Clozapine , 2007 .

[37]  Russell A Poldrack,et al.  Selective corticostriatal dysfunction in schizophrenia: examination of motor and cognitive skill learning. , 2008, Neuropsychology.

[38]  Michael F. Green,et al.  Impaired implicit learning in schizophrenia. , 2008, Neuropsychology.

[39]  Thomas V. Wiecki,et al.  A neurocomputational account of catalepsy sensitization induced by D2 receptor blockade in rats: context dependency, extinction, and renewal , 2009, Psychopharmacology.

[40]  D. Surmeier,et al.  Coordinated expression of dopamine receptors in neostriatal medium spiny neurons. , 1998, Advances in pharmacology.

[41]  S. Snyder,et al.  Antischizophrenic drugs: chronic treatment elevates dopamine receptor binding in brain. , 1977, Science.

[42]  R. V. Van Heertum,et al.  Increased baseline occupancy of D2 receptors by dopamine in schizophrenia. , 2000, Proceedings of the National Academy of Sciences of the United States of America.

[43]  D. Surmeier,et al.  D1 and D2 dopamine-receptor modulation of striatal glutamatergic signaling in striatal medium spiny neurons , 2007, Trends in Neurosciences.

[44]  Samuel M. McClure,et al.  A computational substrate for incentive salience , 2003, Trends in Neurosciences.

[45]  J. Krystal,et al.  Increased striatal dopamine transmission in schizophrenia: confirmation in a second cohort. , 1998, The American journal of psychiatry.

[46]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[47]  P. Greengard,et al.  Regulation of Phosphorylation of the GluR1 AMPA Receptor in the Neostriatum by Dopamine and Psychostimulants In Vivo , 2000, The Journal of Neuroscience.

[48]  Michael J. Frank,et al.  Dynamic Dopamine Modulation in the Basal Ganglia: A Neurocomputational Account of Cognitive Deficits in Medicated and Nonmedicated Parkinsonism , 2005, Journal of Cognitive Neuroscience.

[49]  S. Siris,et al.  Implications of normal brain development for the pathogenesis of schizophrenia. , 1988, Archives of general psychiatry.

[50]  Alexander L. Miller,et al.  Results of phase 3 of the CATIE schizophrenia trial , 2009, Schizophrenia Research.

[51]  G. Fisone,et al.  Looking BAC at striatal signaling: cell-specific analysis in new transgenic mice , 2009, Trends in Neurosciences.

[52]  Jean Addington,et al.  Reliability and validity of a depression rating scale for schizophrenics , 1991, Schizophrenia Research.

[53]  Michael J. Frank,et al.  Single dose of a dopamine agonist impairs reinforcement learning in humans: Evidence from event‐related potentials and computational modeling of striatal‐cortical function , 2009, Human brain mapping.

[54]  Michael X. Cohen,et al.  Neurocomputational models of basal ganglia function in learning, memory and choice , 2009, Behavioural Brain Research.

[55]  R. V. Van Heertum,et al.  Prefrontal Dopamine D1 Receptors and Working Memory in Schizophrenia , 2002, The Journal of Neuroscience.

[56]  M. First,et al.  Structured clinical interview for DSM-IV axis I disorders : SCID-I: clinical version : administration booklet , 1996 .

[57]  M. Laruelle,et al.  Dopamine as the wind of the psychotic fire: new evidence from brain imaging studies , 1999, Journal of psychopharmacology.

[58]  T. Meulemans,et al.  Intact implicit learning in schizophrenia. , 2001, The American journal of psychiatry.

[59]  Michael J. Frank,et al.  A Mechanistic Account of Striatal Dopamine Function in Cognition : Psychopharmacological Studies with Cabergoline and Haloperidol Supplemental Material , 2022 .

[60]  David Wechsler,et al.  Wechsler Memory scale. , 2005 .

[61]  J. Gold,et al.  Effects of contextual processing on visual conditional associative learning in schizophrenia , 2000, Biological Psychiatry.

[62]  J. Gold,et al.  The Wisconsin Card Sorting impairment in schizophrenia is evident in the first four trials , 2008, Schizophrenia Research.

[63]  P. Greengard,et al.  Regulation of phosphorylation of the GluR1 AMPA receptor by dopamine D2 receptors , 2006, Journal of neurochemistry.

[64]  Jennifer A. Mangels,et al.  A Neostriatal Habit Learning System in Humans , 1996, Science.

[65]  E T Rolls,et al.  Representations of pleasant and painful touch in the human orbitofrontal and cingulate cortices. , 2003, Cerebral cortex.

[66]  M. Frank,et al.  Genetic contributions to avoidance-based decisions: striatal D2 receptor polymorphisms , 2009, Neuroscience.

[67]  K. A. Jellinger A Neurodynamic Theory of Schizophrenia (and Related Disorders) , 2009 .

[68]  Michael J. Frank,et al.  Hold your horses: A dynamic computational role for the subthalamic nucleus in decision making , 2006, Neural Networks.

[69]  D. Surmeier,et al.  Coordinated Expression of Dopamine Receptors in Neostriatal Medium Spiny Neurons , 1996, The Journal of Neuroscience.

[70]  P. Seeman,et al.  Dopamine receptors and the dopamine hypothesis of schizophrenia , 1987, Synapse.

[71]  Michael J. Frank,et al.  Genetic triple dissociation reveals multiple roles for dopamine in reinforcement learning , 2007, Proceedings of the National Academy of Sciences.

[72]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[73]  Michael X. Cohen,et al.  A Role for Dopamine in Temporal Decision Making and Reward Maximization in Parkinsonism , 2008, The Journal of Neuroscience.

[74]  M. First,et al.  Structured clinical interview for DSM-IV axis I disorders : SCID-I : clinical version : scoresheet , 1997 .

[75]  J. Waltz,et al.  Reward processing in schizophrenia: a deficit in the representation of value. , 2008, Schizophrenia bulletin.

[76]  M. Frank,et al.  Anatomy of a decision: striato-orbitofrontal interactions in reinforcement learning, decision making, and reversal. , 2006, Psychological review.

[77]  D. Howard,et al.  Implicit learning of visuospatial sequences in schizophrenia. , 2003, Neuropsychology.

[78]  F. Gonon,et al.  Cortical Inputs and GABA Interneurons Imbalance Projection Neurons in the Striatum of Parkinsonian Rats , 2006, The Journal of Neuroscience.

[79]  A. Antal,et al.  Schizophrenics know more than they can tell: probabilistic classification learning in schizophrenia , 2000, Psychological Medicine.