A Neurocomputational Model for Cocaine Addiction

Based on the dopamine hypotheses of cocaine addiction and the assumption of decrement of brain reward system sensitivity after long-term drug exposure, we propose a computational model for cocaine addiction. Utilizing average reward temporal difference reinforcement learning, we incorporate the elevation of basal reward threshold after long-term drug exposure into the model of drug addiction proposed by Redish. Our model is consistent with the animal models of drug seeking under punishment. In the case of nondrug reward, the model explains increased impulsivity after long-term drug exposure. Furthermore, the existence of a blocking effect for cocaine is predicted by our model.

[1]  Robert L Stephens,et al.  Abolished cocaine reward in mice with a cocaine-insensitive dopamine transporter. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[2]  Ben J. A. Kröse,et al.  Learning from delayed rewards , 1995, Robotics Auton. Syst..

[3]  N. Daw,et al.  Reinforcement learning models of the dopamine system and their behavioral implications , 2003 .

[4]  M. Olmstead,et al.  Effects of chronic cocaine on impulsivity: relation to cortical serotonin mechanisms , 2003, Behavioural Brain Research.

[5]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[6]  S. H. Ahmed,et al.  Long-lasting increase in the set point for cocaine self-administration after escalation in rats , 1999, Psychopharmacology.

[7]  T. Robbins,et al.  Neural systems of reinforcement for drug addiction: from actions to habits to compulsion , 2005, Nature Neuroscience.

[8]  J. H. Daruna,et al.  A neurodevelopmental view of impulsivity. , 1993 .

[9]  Robert H Mach,et al.  PET imaging of dopamine D2 receptors during chronic cocaine self-administration in monkeys , 2006, Nature Neuroscience.

[10]  Adam Johnson,et al.  Computing motivation: Incentive salience boosts of drug or appetite states , 2008, Behavioral and Brain Sciences.

[11]  Serge H. Ahmed.  Addiction as Compulsive Reward Prediction , 2004, Science.

[12]  G. Koob,et al.  Plasticity of reward neurocircuitry and the 'dark side' of drug addiction , 2005, Nature Neuroscience.

[13]  David S. Touretzky,et al.  Long-Term Reward Prediction in TD Models of the Dopamine System , 2002, Neural Computation.

[14]  Sham M. Kakade,et al.  Opponent interactions between serotonin and dopamine , 2002, Neural Networks.

[15]  M. Roesch,et al.  Dopamine neurons encode the better option in rats deciding between differently delayed or sized rewards , 2007, Nature Neuroscience.

[16]  H. Gu,et al.  Cocaine does not produce reward in absence of dopamine transporter inhibition , 2009, Neuroreport.

[17]  G. Koob,et al.  Transition to drug addiction: a negative reinforcement model based on an allostatic decrease in reward function , 2005, Psychopharmacology.

[18]  L. Kamin.  Predictability, surprise, attention, and conditioning , 1967 .

[19]  Athina Markou,et al.  Increased motivation for self-administered cocaine after escalated cocaine intake , 2003, Neuroreport.

[20]  Barry Setlow,et al.  Cocaine exposure causes long-term increases in impulsive choice. , 2007, Behavioral neuroscience.

[21]  J. Rabe-Jabłońska,et al.  [Affective disorders in the fourth edition of the classification of mental disorders prepared by the American Psychiatric Association -- diagnostic and statistical manual of mental disorders]. , 1993, Psychiatria polska.

[22]  P. Kalivas,et al.  Drug Addiction as a Pathology of Staged Neuroplasticity , 2008, Neuropsychopharmacology.

[23]  Jadin C. Jackson,et al.  Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. , 2007, Psychological review.

[24]  E. Stein,et al.  Cue-induced cocaine craving: neuroanatomical specificity for drug users and drug stimuli. , 2000, The American journal of psychiatry.

[25]  Sridhar Mahadevan,et al.  Average reward reinforcement learning: Foundations, algorithms, and empirical results , 2004, Machine Learning.

[26]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[27]  A. Grace,et al.  The tonic/phasic model of dopamine system regulation: its relevance for understanding how stimulant abuse can alter basal ganglia function. , 1995, Drug and alcohol dependence.

[28]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[29]  P. Dayan,et al.  Tonic dopamine: opportunity costs and the control of response vigor , 2007, Psychopharmacology.

[30]  Stanislas Dehaene,et al.  A neurocomputational hypothesis for nicotine addiction. , 2006, Proceedings of the National Academy of Sciences of the United States of America.

[31]  L. Panlilio,et al.  Blocking of conditioning to a cocaine-paired stimulus: Testing the hypothesis that cocaine perpetually produces a signal of larger-than-expected reward , 2007, Pharmacology Biochemistry and Behavior.

[32]  S. J. Gatley,et al.  Decreased striatal dopaminergic responsiveness in detoxified cocaine-dependent subjects , 1997, Nature.

[33]  B. Campbell,et al.  Punishment and aversive behavior , 1969 .

[34]  R. Wightman,et al.  Preferential Enhancement of Dopamine Transmission within the Nucleus Accumbens Shell by Cocaine Is Attributable to a Direct Increase in Phasic Dopamine Release Events , 2008, The Journal of Neuroscience.

[35]  Kenji Doya,et al.  What are the computations of the cerebellum, the basal ganglia and the cerebral cortex? , 1999, Neural Networks.

[36]  A. Grace,et al.  The tonic/phasic model of dopamine system regulation and its implications for understanding alcohol and psychostimulant craving. , 2000, Addiction.

[37]  E. Vaadia,et al.  Midbrain dopamine neurons encode decisions for future action , 2006, Nature Neuroscience.

[38]  G F Koob,et al.  Transition from moderate to excessive drug intake: change in hedonic set point. , 1998, Science.

[39]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[40]  Stanley Schachter,et al.  Cocaine decreases self-control in rats: a preliminary report , 2005, Psychopharmacology.

[41]  R. Wightman,et al.  Extinction of Cocaine Self-Administration Reveals Functionally and Temporally Distinct Dopaminergic Signals in the Nucleus Accumbens , 2005, Neuron.

[42]  M. Kreek,et al.  Predictable individual differences in the initiation of cocaine self-administration by rats under extended-access conditions are dose-dependent , 2001, Psychopharmacology.

[43]  K. Doya,et al.  The computational neurobiology of learning and reward , 2006, Current Opinion in Neurobiology.

[44]  Serge H Ahmed.  Neuroscience. Addiction as compulsive reward prediction. , 2004, Science.

[45]  J. Swanson,et al.  Dopamine in drug abuse and addiction: results from imaging studies and treatment implications , 2004, Molecular Psychiatry.

[46]  R. Malenka,et al.  Synaptic plasticity and addiction , 2007, Nature Reviews Neuroscience.

[47]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[48]  Andrew B. Norman,et al.  The compulsion zone: A pharmacological theory of acquired cocaine self-administration , 2006, Brain Research.

[49]  B. Everitt,et al.  Drug Seeking Becomes Compulsive After Prolonged Cocaine Self-Administration , 2004, Science.

[50]  Drake Morgan,et al.  Reduced Dopamine Terminal Function and Insensitivity to Cocaine Following Cocaine Binge Self-Administration and Deprivation , 2005, Neuropsychopharmacology.

[51]  A. Grace.  The tonic/phasic model of dopamine system regulation and its implications for understanding alcohol and psychostimulant craving. , 2000, Addiction.

[52]  Dimitris Samaras,et al.  Is decreased prefrontal cortical sensitivity to monetary reward associated with impaired motivation and self-control in cocaine addiction? , 2007, The American journal of psychiatry.

[53]  David Belin,et al.  Evidence for Addiction-like Behavior in the Rat , 2004, Science.

[54]  Barry J. Everitt,et al.  Compulsive drug seeking by rats under punishment: effects of drug taking history , 2007, Psychopharmacology.

[55]  S. Mahadevan,et al.  Solving Semi-Markov Decision Problems Using Average Reward Reinforcement Learning , 1999 .

[56]  N. Volkow,et al.  The neural basis of addiction: a pathology of motivation and choice. , 2005, The American journal of psychiatry.

[57]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[58]  Caro Lucas,et al.  Individual Differences in Nucleus Accumbens Dopamine Receptors Predict Development of Addiction-Like Behavior: A Computational Approach , 2010, Neural Computation.

[59]  A. Redish,et al.  Addiction as a Computational Process Gone Awry , 2004, Science.

[60]  Francisco S. Melo,et al.  Q -Learning with Linear Function Approximation , 2007, COLT.

[61]  Colin Camerer,et al.  A framework for studying the neurobiology of value-based decision making , 2008, Nature Reviews Neuroscience.