Reduction of Pavlovian Bias in Schizophrenia: Enhanced Effects in Clozapine-Administered Patients

The negative symptoms of schizophrenia (SZ) are associated with a pattern of reinforcement learning (RL) deficits likely related to degraded representations of reward values. However, the RL tasks used to date have required active responses to both reward and punishing stimuli. Pavlovian biases have been shown to affect performance on these tasks through invigoration of action to reward and inhibition of action to punishment, and may be partially responsible for the effects found in patients. Forty-five patients with schizophrenia and 30 demographically-matched controls completed a four-stimulus reinforcement learning task that crossed action (“Go” or “NoGo”) and the valence of the optimal outcome (reward or punishment-avoidance), such that all combinations of action and outcome valence were tested. Behaviour was modelled using a six-parameter RL model and EEG was simultaneously recorded. Patients demonstrated a reduction in Pavlovian performance bias that was evident in a reduced Go bias across the full group. In a subset of patients administered clozapine, the reduction in Pavlovian bias was enhanced. The reduction in Pavlovian bias in SZ patients was accompanied by feedback processing differences at the time of the P3a component. The reduced Pavlovian bias in patients is suggested to be due to reduced fidelity in the communication between striatal regions and frontal cortex. It may also partially account for previous findings of poorer “Go-learning” in schizophrenia where “Go” responses or Pavlovian consistent responses are required for optimal performance. An attenuated P3a component dynamic in patients is consistent with a view that deficits in operant learning are due to impairments in adaptively using feedback to update representations of stimulus value.

[1]  Rolf Verleger,et al.  P3b: Towards some decision about memory , 2008, Clinical Neurophysiology.

[2]  M. Frank,et al.  Frontal theta as a mechanism for cognitive control , 2014, Trends in Cognitive Sciences.

[3]  A. Levey,et al.  The role of M1 muscarinic receptor agonism of N-desmethylclozapine in the unique clinical effects of clozapine , 2004, Psychopharmacology.

[4]  J. Kruschke Doing Bayesian Data Analysis: A Tutorial with R and BUGS , 2010 .

[5]  P. Dayan,et al.  Action controls dopaminergic enhancement of reward representations , 2012, Proceedings of the National Academy of Sciences.

[6]  W. Schultz Updating dopamine reward signals , 2013, Current Opinion in Neurobiology.

[7]  T. Crow,et al.  Regional deficits in brain volume in schizophrenia: a meta-analysis of voxel-based morphometry studies. , 2005, The American journal of psychiatry.

[8]  Anne G E Collins,et al.  Opponent actor learning (OpAL): modeling interactive effects of striatal dopamine on reinforcement learning and choice incentive. , 2014, Psychological review.

[9]  James A. Waltz,et al.  Probabilistic reversal learning impairments in schizophrenia: Further evidence of orbitofrontal dysfunction , 2007, Schizophrenia Research.

[10]  G. Hajcak,et al.  Impaired neural response to internal but not external feedback in schizophrenia , 2011, Psychological Medicine.

[11]  Clay B. Holroyd,et al.  Human Neuroscience Original Research Article Dissociation of Response and Feedback Negativity in Schizophrenia: Electrophysiological and Computational Evidence for a Deficit in the Representation of Value , 2022 .

[12]  W. Hershberger An approach through the looking-glass , 1986 .

[13]  A. Simmons,et al.  Emotional decision-making and its dissociable components in schizophrenia and schizoaffective disorder: A behavioural and MRI investigation , 2008, Neuropsychologia.

[14]  M. Minzenberg,et al.  Meta-analysis of 41 functional neuroimaging studies of executive function in schizophrenia. , 2009, Archives of general psychiatry.

[15]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[16]  Adrian G. Fischer,et al.  Neural mechanisms and temporal dynamics of performance monitoring , 2014, Trends in Cognitive Sciences.

[17]  D. Javitt,et al.  Inhibition of System A-mediated glycine transport in cortical synaptosomes by therapeutic concentrations of clozapine: implications for mechanisms of action , 2005, Molecular Psychiatry.

[18]  Pavel Mohr,et al.  LORETA Functional Imaging in Antipsychotic-Naive and Olanzapine-, Clozapine- and Risperidone-Treated Patients with Schizophrenia , 2008, Neuropsychobiology.

[19]  Rolf Verleger,et al.  Testing the stimulus-to-response bridging function of the oddball-P3 by delayed response signals and residue iteration decomposition (RIDE) , 2014, NeuroImage.

[20]  Michael X. Cohen,et al.  Single-Trial Regression Elucidates the Role of Prefrontal Theta Oscillations in Response Conflict , 2011, Front. Psychology.

[21]  T. Goldberg,et al.  Auditory working memory and Wisconsin Card Sorting Test performance in schizophrenia. , 1997, Archives of general psychiatry.

[22]  T. Sigmundsson,et al.  Structural abnormalities in frontal, temporal, and limbic regions and interconnecting white matter tracts in schizophrenic patients with prominent negative symptoms. , 2001, The American journal of psychiatry.

[23]  A. Meyer-Lindenberg,et al.  Ventral striatal activation during attribution of stimulus saliency and reward anticipation is correlated in unmedicated first episode schizophrenia patients , 2012, Schizophrenia Research.

[24]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[25]  S. Kapur,et al.  Does fast dissociation from the dopamine d(2) receptor explain the action of atypical antipsychotics?: A new hypothesis. , 2001, The American journal of psychiatry.

[26]  G. Elmer,et al.  Disruption of conditioned reward association by typical and atypical antipsychotics , 2010, Pharmacology Biochemistry and Behavior.

[27]  Michael X. Cohen,et al.  Frontal Oscillatory Dynamics Predict Feedback Learning and Action Adjustment , 2011, Journal of Cognitive Neuroscience.

[28]  D. Tucker,et al.  Regulating action: alternating activation of midline frontal and motor cortical networks , 2001, Clinical Neurophysiology.

[29]  Clay B. Holroyd,et al.  The feedback correct-related positivity: sensitivity of the event-related brain potential to unexpected positive feedback. , 2008, Psychophysiology.

[30]  J. Kruschke Bayesian estimation supersedes the t test. , 2013, Journal of experimental psychology. General.

[31]  R. Murray,et al.  Dopamine synthesis capacity in patients with treatment-resistant schizophrenia. , 2012, The American journal of psychiatry.

[32]  Josep Marco-Pallarés,et al.  The role of beta-gamma oscillations in unexpected rewards processing , 2012, NeuroImage.

[33]  B. Mowry,et al.  Structured interview for DSM-IV personality: SIDP-IV , 1998 .

[34]  Anthony A. Grace,et al.  Dopamine-cell depolarization block as a model for the therapeutic actions of antipsychotic drugs , 1997, Trends in Neurosciences.

[35]  John J. B. Allen,et al.  Prelude to and Resolution of an Error: EEG Phase Synchrony Reveals Cognitive Control Dynamics during Action Monitoring , 2009, The Journal of Neuroscience.

[36]  J. Waltz,et al.  A review of reward processing and motivational impairment in schizophrenia. , 2014, Schizophrenia bulletin.

[37]  Markus Ullsperger,et al.  Neuropharmacology of performance monitoring , 2009, Neuroscience & Biobehavioral Reviews.

[38]  A. Malhotra,et al.  Spectrum of EEG abnormalities during clozapine treatment. , 1994, Electroencephalography and Clinical Neurophysiology.

[39]  Thomas E. Nichols,et al.  luster-based computational methods for mass univariate analyses f event-related brain potentials / fields : A simulation study , 2022 .

[40]  Yves Chaput,et al.  Quantified EEG changes associated with a positive clinical response to clozapine in schizophrenia , 1995, Progress in Neuro-Psychopharmacology and Biological Psychiatry.

[41]  H. Meltzer,et al.  The role of serotonin receptors in the action of atypical antipsychotic drugs. , 2011, Current opinion in pharmacology.

[42]  Anne G E Collins,et al.  Working Memory Contributions to Reinforcement Learning Impairments in Schizophrenia , 2014, The Journal of Neuroscience.

[43]  Melissa J. Green,et al.  Task-related fronto-striatal functional connectivity during working memory performance in schizophrenia , 2013, Schizophrenia Research.

[44]  Cameron S. Carter,et al.  Impaired Prefrontal-Basal Ganglia Functional Connectivity and Substantia Nigra Hyperactivity in Schizophrenia , 2013, Biological Psychiatry.

[45]  M. Kubicki,et al.  Functional and anatomical connectivity abnormalities in left inferior frontal gyrus in schizophrenia , 2009, Human brain mapping.

[46]  Aapo Hyvärinen,et al.  Learning Natural Image Structure with a Horizontal Product Model , 2009, ICA.

[47]  Terrence J. Sejnowski,et al.  Enhanced detection of artifacts in EEG data using higher-order statistics and independent component analysis , 2007, NeuroImage.

[48]  W. Schultz Activity of dopamine neurons in the behaving primate , 1992 .

[49]  Arnaud Delorme,et al.  EEGLAB: an open source toolbox for analysis of single-trial EEG dynamics including independent component analysis , 2004, Journal of Neuroscience Methods.

[50]  B. Balleine,et al.  Differential dependence of Pavlovian incentive motivation and instrumental incentive learning processes on dopamine signaling. , 2011, Learning & memory.

[51]  G. Juckel,et al.  Reward Feedback Alterations in Unmedicated Schizophrenia Patients: Relevance for Delusions , 2009, Biological Psychiatry.

[52]  Bhaskar D. Rao,et al.  Modeling and Estimation of Dependent Subspaces with Non-radially Symmetric and Skewed Densities , 2007, ICA.

[53]  Adrian R. Willoughby,et al.  The Medial Frontal Cortex and the Rapid Processing of Monetary Gains and Losses , 2002, Science.

[54]  Ramin Khatami,et al.  Advanced EEG analysis using threshold-free cluster-enhancement and non-parametric statistics , 2013, NeuroImage.

[55]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[56]  U. Habel,et al.  Effects of serotonin depletion on punishment processing in the orbitofrontal and anterior cingulate cortices of healthy women , 2015, European Neuropsychopharmacology.

[57]  Michael J Frank,et al.  Negative symptoms and the failure to represent the expected reward value of actions: behavioral and computational modeling evidence. , 2012, Archives of general psychiatry.

[58]  M. Frank,et al.  Deficits in Positive Reinforcement Learning and Uncertainty-Driven Exploration Are Associated with Distinct Aspects of Negative Symptoms in Schizophrenia , 2011, Biological Psychiatry.

[59]  Clay B. Holroyd,et al.  Learning-related changes in brain activity following errors and performance feedback in schizophrenia , 2008, Schizophrenia Research.

[60]  S. Kelly,et al.  The classic P300 encodes a build‐to‐threshold decision variable , 2015, The European journal of neuroscience.

[61]  Markus Ullsperger,et al.  Real and Fictive Outcomes Are Processed Differently but Converge on a Common Adaptive Mechanism , 2013, Neuron.

[62]  Ian W. Eisenberg,et al.  Frontal Theta Overrides Pavlovian Learning Biases , 2013, The Journal of Neuroscience.

[63]  G. Price,et al.  The impact of clozapine on electrophysiological features: how can we utilize the findings? , 2002, International journal of psychiatry in clinical practice.

[64]  Gerd Wagner,et al.  Altered activation in association with reward-related trial-and-error learning in patients with schizophrenia , 2010, NeuroImage.

[65]  K. R. Ridderinkhof,et al.  Medial frontal cortex and response conflict: Evidence from human intracranial EEG and medial frontal cortex lesion , 2008, Brain Research.

[66]  J. Polich Updating P300: An integrative theory of P3a and P3b , 2007, Clinical Neurophysiology.

[67]  Raymond J. Dolan,et al.  Go and no-go learning in reward and punishment: Interactions between affect and effect , 2012, NeuroImage.

[68]  Tomas Palenicek,et al.  Mechanism of Action of Atypical Antipsychotic Drugs and the Neurobiology of Schizophrenia , 2006, CNS drugs.

[69]  H. Meltzer Attention Must Be Paid: The Association of Plasma Clozapine/NDMC Ratio With Working Memory. , 2015, The American journal of psychiatry.

[70]  E. Rolls,et al.  Computational models of schizophrenia and dopamine modulation in the prefrontal cortex , 2008, Nature Reviews Neuroscience.

[71]  Clay B. Holroyd,et al.  The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. , 2002, Psychological review.

[72]  V. Knott,et al.  Quantitative EEG in schizophrenia and in response to acute and chronic clozapine treatment , 2001, Schizophrenia Research.

[73]  James F. Cavanagh,et al.  Frontal theta links prediction errors to behavioral adaptation in reinforcement learning , 2010, NeuroImage.

[74]  J. Waltz,et al.  Reward processing in schizophrenia: a deficit in the representation of value. , 2008, Schizophrenia bulletin.

[75]  Michael X. Cohen,et al.  Midfrontal conflict-related theta-band power reflects neural oscillations that predict behavior. , 2013, Journal of neurophysiology.

[76]  Arno Villringer,et al.  Dysfunction of ventral striatal reward prediction in schizophrenia , 2006, NeuroImage.

[77]  T. Goldberg,et al.  Further evidence for dementia of the prefrontal type in schizophrenia? A controlled study of teaching the Wisconsin Card Sorting Test. , 1987, Archives of general psychiatry.

[78]  Maria F. Sassano,et al.  Automated design of ligands to polypharmacological profiles , 2012, Nature.