Dopamine modulates novelty seeking behavior during decision making.

Novelty seeking refers to the tendency of humans and animals to explore novel and unfamiliar stimuli and environments. The idea that dopamine modulates novelty seeking is supported by evidence that novel stimuli excite dopamine neurons and activate brain regions receiving dopaminergic input. In addition, dopamine is shown to drive exploratory behavior in novel environments. It is not clear whether dopamine promotes novelty seeking when it is framed as the decision to explore novel options versus the exploitation of familiar options. To test this hypothesis, we administered systemic injections of saline or GBR-12909, a selective dopamine transporter (DAT) inhibitor, to monkeys and assessed their novelty seeking behavior during a probabilistic decision making task. The task involved pseudorandom introductions of novel choice options. This allowed monkeys the opportunity to explore novel options or to exploit familiar options that they had already sampled. We found that DAT blockade increased the monkeys' preference for novel options. A reinforcement learning (RL) model fit to the monkeys' choice data showed that increased novelty seeking after DAT blockade was driven by an increase in the initial value the monkeys assigned to novel options. However, blocking DAT did not modulate the rate at which the monkeys learned which cues were most predictive of reward or their tendency to exploit that knowledge. These data demonstrate that dopamine enhances novelty-driven value and imply that excessive novelty seeking-characteristic of impulsivity and behavioral addictions-might be caused by increases in dopamine, stemming from less reuptake.

[1]  HighWire Press Philosophical Transactions of the Royal Society of London , 1781, The London Medical Journal.

[2]  J. Neter,et al.  Applied linear statistical models : regression, analysis of variance, and experimental designs , 1974 .

[3]  J. Gittins Bandit processes and dynamic allocation indices , 1979 .

[4]  O. Rafaelsen,et al.  A Tolerance Study of Single and Multiple Dosing of the Selective Dopamine Uptake Inhibitor GBR 12909 in Healthy Subjects , 1990, International clinical psychopharmacology.

[5]  Patricia S. Goldman-Rakic,et al.  Viewing preferences of rhesus monkeys related to memory for complex pictures, colours and faces , 1994, Behavioural Brain Research.

[6]  T. Wills,et al.  Novelty seeking, risk taking, and related constructs as predictors of adolescent substance use: an application of Cloninger's theory. , 1994, Journal of substance abuse.

[7]  Joel R. Levin,et al.  A controlled, powerful multiple-comparison strategy for several situations. , 1994 .

[8]  K. Rice,et al.  Effects of dopamine reuptake inhibitors on food- and cocaine-maintained responding: I. Dependence on unit dose of cocaine. , 1995 .

[9]  K. Rice,et al.  Effects of dopamine reuptake inhibitors on food- and cocaine-maintained responding: II. Comparisons with other drugs and repeated administrations. , 1995 .

[10]  L J Porrino,et al.  Cocaine alters cerebral metabolism within the ventral striatum and limbic cortex of monkeys , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[11]  T. Nokes,et al.  Intrinsic reinforcing properties of putatively neutral stimuli in an instrumental two-lever discrimination task , 1996 .

[12]  D. Wong,et al.  Doses of GBR12909 that suppress cocaine self‐administration in non‐human primates substantially occupy dopamine transporters as measured by [11C] WIN35,428 PET scans , 1999, Synapse.

[13]  M. Low,et al.  Dopamine D4 Receptor-Knock-Out Mice Exhibit Reduced Exploration of Novel Stimuli , 1999, The Journal of Neuroscience.

[14]  L. Howell,et al.  Cocaine-induced changes in extracellular dopamine determined by microdialysis in awake squirrel monkeys , 2000, Psychopharmacology.

[15]  Hideo Tsukada,et al.  Dose–response and duration effects of acute administrations of cocaine and GBR12909 on dopamine synthesis and transporter in the conscious monkey brain: PET studies combined with microdialysis , 2000, Brain Research.

[16]  J. Horvitz Mesolimbocortical and nigrostriatal dopamine responses to salient non-reward events , 2000, Neuroscience.

[17]  A. Sampson,et al.  Dopamine transporter immunoreactivity in monkey cerebral cortex: Regional, laminar, and ultrastructural localization , 2001, The Journal of comparative neurology.

[18]  R. Hen,et al.  Hyperactivity and impaired response habituation in hyperdopaminergic mice. , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[19]  Peter Dayan,et al.  Dopamine: generalization and bonuses , 2002, Neural Networks.

[20]  A. Galli,et al.  Regulation of dopamine transporter function and plasma membrane expression by dopamine, amphetamine, and cocaine. , 2003, European journal of pharmacology.

[21]  J. Seamans,et al.  The principal features and mechanisms of dopamine modulation in the prefrontal cortex , 2004, Progress in Neurobiology.

[22]  W. Newsome,et al.  Matching Behavior and the Representation of Value in the Parietal Cortex , 2004, Science.

[23]  Michael J. Frank,et al.  By Carrot or by Stick: Cognitive Reinforcement Learning in Parkinsonism , 2004, Science.

[24]  E. Butelman,et al.  Genetic influences on impulsivity, risk taking, stress responsivity and vulnerability to drug abuse and addiction , 2005, Nature Neuroscience.

[25]  A. Mitz A liquid-delivery device that provides precise reward control for neurophysiological and behavioral experiments , 2005, Journal of Neuroscience Methods.

[26]  R. Palmiter,et al.  Distinguishing whether dopamine regulates liking, wanting, and/or learning about rewards. , 2005, Behavioral neuroscience.

[27]  N. Bunzeck,et al.  Absolute Coding of Stimulus Novelty in the Human Substantia Nigra/VTA , 2006, Neuron.

[28]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[29]  P. Dayan,et al.  Tonic dopamine: opportunity costs and the control of response vigor , 2007, Psychopharmacology.

[30]  Thomas T. Hills Animal Foraging and the Evolution of Goal-Directed Cognition , 2006, Cogn. Sci..

[31]  N. Volkow,et al.  Structural and metabolic brain changes in the striatum associated with methamphetamine abuse. , 2007, Addiction.

[32]  R. Ebstein,et al.  Relationship between dopamine system genes and extraversion and novelty seeking , 2007, Neuroscience and Behavioral Physiology.

[33]  Angela J. Yu,et al.  Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[34]  N. Daw,et al.  Striatal Activity Underlies Novelty-Based Choice in Humans , 2008, Neuron.

[35]  Emad N. Eskandar,et al.  A flexible software tool for temporally-precise behavioral control in Matlab , 2008, Journal of Neuroscience Methods.

[36]  John A. Dani,et al.  Controls of Tonic and Phasic Dopamine Transmission in the Dorsal and Ventral Striatum , 2009, Molecular Pharmacology.

[37]  Andrew J Lees,et al.  Intact Reward Learning but Elevated Delay Discounting in Parkinson's Disease Patients With Impulsive-Compulsive Spectrum Behaviors , 2010, Neuropsychopharmacology.

[38]  Ethan S. Bromberg-Martin,et al.  Dopamine in Motivational Control: Rewarding, Aversive, and Alerting , 2010, Neuron.

[39]  Nathaniel Daw,et al.  Behavioral Neuroscience , 2022 .

[40]  Mark Slifstein,et al.  Striatal and extrastriatal dopamine release measured with PET and [18F] fallypride , 2010, Synapse.

[41]  Thomas E. Hazy,et al.  Neural mechanisms of acquired phasic dopamine responses in learning , 2010, Neuroscience & Biobehavioral Reviews.

[42]  B. Averbeck,et al.  Risk and learning in impulsive and nonimpulsive patients with Parkinson's disease , 2010, Movement disorders : official journal of the Movement Disorder Society.

[43]  P. Dayan,et al.  A common mechanism for adaptive scaling of reward and novelty , 2010, Human brain mapping.

[44]  Greg O. Horne,et al.  Controlling low-level image properties: The SHINE toolbox , 2010, Behavior research methods.

[45]  A. Strafella,et al.  Reduced dopamine transporter density in the ventral striatum of patients with Parkinson's disease and pathological gambling , 2010, Neurobiology of Disease.

[46]  B. Averbeck,et al.  Novelty seeking behaviour in Parkinson's disease , 2011, Neuropsychologia.

[47]  T. Robbins,et al.  Impulsivity, Compulsivity, and Top-Down Cognitive Control , 2011, Neuron.

[48]  O. Eriksson,et al.  Assessment of receptor occupancy-over-time of two dopamine transporter inhibitors by [11C]CIT and target controlled infusion , 2011, Upsala journal of medical sciences.

[49]  John M. Pearson,et al.  Neuronal basis of sequential foraging decisions in a patchy environment , 2011, Nature Neuroscience.

[50]  PJ Harrison,et al.  Modulation of hippocampal dopamine metabolism and hippocampal-dependent cognitive function by catechol-O-methyltransferase inhibition , 2012, Journal of psychopharmacology.

[51]  Timothy E. J. Behrens,et al.  Neural Mechanisms of Foraging , 2012, Science.

[52]  Claude Comtat,et al.  Striatal and extrastriatal dopamine transporter in cannabis and tobacco addiction: a high‐resolution PET study , 2012, Addiction biology.

[53]  Therese van Amelsvoort,et al.  The Detection of Novelty Relies on Dopaminergic Signaling: Evidence from Apomorphine's Impact on the Novelty N2 , 2013, PloS one.

[54]  E. Murray,et al.  The drive to strive: goal generation based on current needs , 2013, Front. Neurosci..

[55]  B. Averbeck,et al.  Uncertainty about mapping future actions into rewards may underlie performance on multiple measures of impulsivity in behavioral addiction: evidence from Parkinson's disease. , 2013, Behavioral neuroscience.