Dopamine reward prediction-error signalling: a two-component response

Environmental stimuli and objects, including rewards, are often processed sequentially in the brain. Recent work suggests that the phasic dopamine reward prediction-error response follows a similar sequential pattern. An initial brief, unselective and highly sensitive increase in activity unspecifically detects a wide range of environmental stimuli, then quickly evolves into the main response component, which reflects subjective reward value and utility. This temporal evolution allows the dopamine reward prediction-error signal to optimally combine speed and accuracy.

[1]  W. Schultz,et al.  Responses of nigrostriatal dopamine neurons to high-intensity somatosensory stimulation in the anesthetized monkey. , 1987, Journal of neurophysiology.

[2]  M. Rothschild,et al.  Increasing risk: I. A definition , 1970 .

[3]  S. Ikemoto,et al.  Similar Roles of Substantia Nigra and Ventral Tegmental Dopamine Neurons in Reward and Aversion , 2014, The Journal of Neuroscience.

[4]  Tatsuo K Sato,et al.  Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons , 2003, The Journal of Neuroscience.

[5]  W. Schultz,et al.  Responses of monkey dopamine neurons during learning of behavioral reactions. , 1992, Journal of neurophysiology.

[6]  Anna E. Ipata,et al.  Feature attention evokes task-specific pattern selectivity in V4 neurons , 2012, Proceedings of the National Academy of Sciences.

[7]  W. Schultz,et al.  Responses of monkey dopamine neurons to reward and conditioned stimuli during successive steps of learning a delayed response task , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[8]  Joshua L. Jones,et al.  Phasic Nucleus Accumbens Dopamine Release Encodes Effort- and Delay-Related Costs , 2010, Biological Psychiatry.

[9]  M. Shadlen,et al.  Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task , 2002, The Journal of Neuroscience.

[10]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[11]  M. Roitman,et al.  Nucleus Accumbens Neurons Are Innately Tuned for Rewarding and Aversive Taste Stimuli, Encode Their Predictors, and Are Linked to Motor Output , 2005, Neuron.

[12]  J. Hegdé Time course of visual perception: Coarse-to-fine processing and beyond , 2008, Progress in Neurobiology.

[13]  A. Cooper,et al.  Predictive Reward Signal of Dopamine Neurons , 2011 .

[14]  M. Platt,et al.  Risk-sensitive neurons in macaque posterior cingulate cortex , 2005, Nature Neuroscience.

[15]  R. Wightman,et al.  Aversive stimulus differentially triggers subsecond dopamine release in reward regions , 2012, Neuroscience.

[16]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[17]  K. Doya,et al.  Representation of Action-Specific Reward Values in the Striatum , 2005, Science.

[18]  L. J. Savage,et al.  The Utility Analysis of Choices Involving Risk , 1948, Journal of Political Economy.

[19]  R. Solomon,et al.  An Opponent-Process Theory of Motivation , 1978 .

[20]  J. Kagel,et al.  Economic Choice Theory: An Experimental Analysis of Animal Behavior , 1995 .

[21]  William R. Stauffer,et al.  Dopamine prediction error responses integrate subjective value from different reward dimensions , 2014, Proceedings of the National Academy of Sciences.

[22]  L. Wilbrecht,et al.  Transient stimulation of distinct subpopulations of striatal neurons mimics changes in action value , 2012, Nature Neuroscience.

[23]  L. J. Savage,et al.  The Foundations of Statistics , 1955 .

[24]  Daeyeol Lee,et al.  Heterogeneous Coding of Temporally Discounted Values in the Dorsal and Ventral Striatum during Intertemporal Choice , 2011, Neuron.

[25]  E. Oleson,et al.  Subsecond Dopamine Release in the Nucleus Accumbens Predicts Conditioned Punishment and Its Successful Avoidance , 2012, The Journal of Neuroscience.

[26]  Rajesh P. N. Rao,et al.  Predictive coding in the visual cortex: a functional interpretation of some extra-classical receptive-field effects. , 1999 .

[27]  John Anderson The foundations of IN , 2002 .

[28]  Takeo Watanabe,et al.  Temporally Extended Dopamine Responses to Perceptually Demanding Reward-Predictive Stimuli , 2010, The Journal of Neuroscience.

[29]  Liqun Luo,et al.  Diversity of Transgenic Mouse Models for Selective Targeting of Midbrain Dopamine Neurons , 2015, Neuron.

[30]  W. Schultz,et al.  Neuronal activity in monkey striatum related to the expectation of predictable environmental events. , 1992, Journal of neurophysiology.

[31]  N. Mackintosh The psychology of animal learning , 1974 .

[32]  J. Fuster Unit activity in prefrontal cortex during delayed-response performance: neuronal correlates of transient memory. , 1973, Journal of neurophysiology.

[33]  S. Cragg,et al.  Substance P Weights Striatal Dopamine Transmission Differently within the Striosome-Matrix Axis , 2015, The Journal of Neuroscience.

[34]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[35]  P. Garris,et al.  Different effects of cocaine and nomifensine on dopamine uptake in the caudate-putamen and nucleus accumbens. , 1995, The Journal of pharmacology and experimental therapeutics.

[36]  E. Vaadia,et al.  Midbrain Dopaminergic Neurons and Striatal Cholinergic Interneurons Encode the Difference between Reward and Aversive Events at Different Epochs of Probabilistic Classical Conditioning Trials , 2008, The Journal of Neuroscience.

[37]  S. Kapur Psychosis as a state of aberrant salience: a framework linking biology, phenomenology, and pharmacology in schizophrenia. , 2003, The American journal of psychiatry.

[38]  Anne E Carpenter,et al.  Neuron-type specific signals for reward and punishment in the ventral tegmental area , 2011, Nature.

[39]  A. Grace,et al.  Acute and chronic haloperidol treatment: comparison of effects on nigral dopaminergic cell activity. , 1978, Life sciences.

[40]  John H. R. Maunsell,et al.  Attentional modulation of visual motion processing in cortical areas MT and MST , 1996, Nature.

[41]  M. Feenstra,et al.  Dopamine efflux in nucleus accumbens shell and core in response to appetitive classical conditioning , 2003, The European journal of neuroscience.

[42]  J. Bargas,et al.  D2 Dopamine Receptors in Striatal Medium Spiny Neurons Reduce L-Type Ca2+ Currents and Excitability via a Novel PLCβ1–IP3–Calcineurin-Signaling Cascade , 2000, The Journal of Neuroscience.

[43]  R. Schubert,et al.  Rationality on the rise: Why relative risk aversion increases with stake size , 2009 .

[44]  Kelly R. Tan,et al.  GABA Neurons of the VTA Drive Conditioned Place Aversion , 2012, Neuron.

[45]  W. Schultz,et al.  Preferential activation of midbrain dopamine neurons by appetitive rather than aversive stimuli , 1996, Nature.

[46]  Andrew M. J. Young,et al.  Increased extracellular dopamine in nucleus accumbens in response to unconditioned and conditioned aversive stimuli: studies using 1 min microdialysis in rats , 2004, Journal of Neuroscience Methods.

[47]  P. Roelfsema,et al.  Simultaneous selection by object-based attention in visual and frontal cortex , 2014, Proceedings of the National Academy of Sciences.

[48]  Andrea Anzalone,et al.  Dual Control of Dopamine Synthesis and Release by Presynaptic and Postsynaptic Dopamine D2 Receptors , 2012, The Journal of Neuroscience.

[49]  W. Schultz Multiple dopamine functions at different time courses. , 2007, Annual review of neuroscience.

[50]  J. Pearce,et al.  A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980 .

[51]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[52]  T. Robbins,et al.  The neuropsychopharmacology of fronto-executive function: monoaminergic modulation. , 2009, Annual review of neuroscience.

[53]  P. Greengard,et al.  Dopamine and cAMP-Regulated Phosphoprotein 32 kDa Controls Both Striatal Long-Term Depression and Long-Term Potentiation, Opposing Forms of Synaptic Plasticity , 2000, The Journal of Neuroscience.

[54]  J. Gray,et al.  Increased dopamine release in vivo in nucleus accumbens and caudate nucleus of the rat during drinking: A microdialysis study , 1992, Neuroscience.

[55]  Minryung R. Song,et al.  Diversity and Homogeneity in Responses of Midbrain Dopamine Neurons , 2013, The Journal of Neuroscience.

[56]  K. Deisseroth,et al.  Phasic Firing in Dopaminergic Neurons Is Sufficient for Behavioral Conditioning , 2009, Science.

[57]  Wolfram Schultz,et al.  Reward Contexts Extend Dopamine Signals to Unrewarded Stimuli , 2014, Current Biology.

[58]  E. Vaadia,et al.  Coincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons , 2004, Neuron.

[59]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[60]  J. Bargas,et al.  D1 Receptor Activation Enhances Evoked Discharge in Neostriatal Medium Spiny Neurons by Modulating an L-Type Ca2+ Conductance , 1997, The Journal of Neuroscience.

[61]  R. Solomon,et al.  An opponent-process theory of motivation. I. Temporal dynamics of affect. , 1974, Psychological review.

[62]  Mark G. Baxter,et al.  The Rostromedial Tegmental Nucleus (RMTg), a GABAergic Afferent to Midbrain Dopamine Neurons, Encodes Aversive Stimuli and Inhibits Motor Responses , 2009, Neuron.

[63]  David M. Kreps,et al.  A Course in Microeconomic Theory , 2020 .

[64]  M. H. Joseph,et al.  Conditioned appetitive stimulus increases extracellular dopamine in the nucleus accumbens of the rat , 2002, The European journal of neuroscience.

[65]  D J Henry,et al.  Electrophysiological effects of cocaine in the mesoaccumbens dopamine system: repeated administration. , 1989, The Journal of pharmacology and experimental therapeutics.

[66]  Aneesha Badrinarayan,et al.  Aversive Stimuli Differentially Modulate Real-Time Dopamine Transmission Dynamics within the Nucleus Accumbens Core and Shell , 2012, The Journal of Neuroscience.

[67]  S. Thorpe,et al.  The orbitofrontal cortex: Neuronal activity in the behaving monkey , 2004, Experimental Brain Research.

[68]  T. Jay,et al.  Essential Role of D1 But Not D2 Receptors in the NMDA Receptor-Dependent Long-Term Potentiation at Hippocampal-Prefrontal Cortex Synapses In Vivo , 2000, The Journal of Neuroscience.

[69]  W. Schultz Neuronal Reward and Decision Signals: From Theories to Data. , 2015, Physiological reviews.

[70]  B. Averbeck,et al.  Action Selection and Action Value in Frontal-Striatal Circuits , 2012, Neuron.

[71]  Kathy Dujardin,et al.  Effects of stimulus-driven and goal-directed attention on prepulse inhibition of the cortical responses to an auditory pulse , 2014, Clinical Neurophysiology.

[72]  B. Jacobs,et al.  Behavioral correlates of dopaminergic unit activity in freely moving cats , 1983, Brain Research.

[73]  G. Stuber,et al.  Activation of VTA GABA Neurons Disrupts Reward Consumption , 2012, Neuron.

[74]  K. Deisseroth,et al.  Striatal Dopamine Release Is Triggered by Synchronized Activity in Cholinergic Interneurons , 2012, Neuron.

[75]  R. Palmiter,et al.  Disruption of NMDAR-dependent burst firing by dopamine neurons provides selective assessment of phasic dopamine-dependent behavior , 2009, Proceedings of the National Academy of Sciences.

[76]  栁下 祥 A critical time window for dopamine actions on the structural plasticity of dendritic spines , 2016 .

[77]  Christopher J. Peck,et al.  The primate amygdala combines information about space and value , 2013, Nature Neuroscience.

[78]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[79]  T. Womelsdorf,et al.  Dynamic shifts of visual receptive fields in cortical area MT by spatial attention , 2006, Nature Neuroscience.

[80]  M. Shadlen,et al.  Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque , 1999, Nature Neuroscience.

[81]  R. Luce,et al.  Individual Choice Behavior: A Theoretical Analysis. , 1960 .

[82]  W. Schultz Responses of midbrain dopamine neurons to behavioral trigger stimuli in the monkey. , 1986, Journal of neurophysiology.

[83]  K. Wilcox,et al.  Stimulation of the lateral habenula inhibits dopamine-containing neurons in the substantia nigra and ventral tegmental area of the rat , 1986, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[84]  K. Deisseroth,et al.  Input-specific control of reward and aversion in the ventral tegmental area , 2012, Nature.

[85]  J. Wickens,et al.  A cellular mechanism of reward-related learning , 2001, Nature.

[86]  J. Hollerman,et al.  Influence of reward expectation on behavior-related neuronal activity in primate striatum. , 1998, Journal of neurophysiology.

[87]  Simon Hong,et al.  Negative Reward Signals from the Lateral Habenula to Dopamine Neurons Are Mediated by Rostromedial Tegmental Nucleus in Primates , 2011, The Journal of Neuroscience.

[88]  Dario L. Ringach,et al.  Dynamics of orientation tuning in macaque primary visual cortex , 1997, Nature.

[89]  Talia N. Lerner,et al.  Intact-Brain Analyses Reveal Distinct Information Carried by SNc Dopamine Subcircuits , 2015, Cell.

[90]  R. Wightman,et al.  Extinction of Cocaine Self-Administration Reveals Functionally and Temporally Distinct Dopaminergic Signals in the Nucleus Accumbens , 2005, Neuron.

[91]  J. Horvitz,et al.  Burst activity of ventral tegmental dopamine neurons is elicited by sensory stimuli in the awake cat , 1997, Brain Research.

[92]  Josiah R. Boivin,et al.  A Causal Link Between Prediction Errors, Dopamine Neurons and Learning , 2013, Nature Neuroscience.

[93]  Ethan S. Bromberg-Martin,et al.  Lateral habenula neurons signal errors in the prediction of reward information , 2011, Nature Neuroscience.

[94]  W. Schultz,et al.  Influence of Reward Delays on Responses of Dopamine Neurons , 2008, The Journal of Neuroscience.

[95]  P. Greengard,et al.  Dichotomous Dopaminergic Control of Striatal Synaptic Plasticity , 2008, Science.

[96]  Veit Stuphorn,et al.  Supplementary Eye Field Encodes Reward Prediction Error , 2012, The Journal of Neuroscience.

[97]  M. Ungless,et al.  Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli , 2009, Proceedings of the National Academy of Sciences.

[98]  William R. Stauffer,et al.  Dopamine Reward Prediction Error Responses Reflect Marginal Utility , 2014, Current Biology.

[99]  Pieter R. Roelfsema,et al.  Different Processing Phases for Features, Figures, and Selective Attention in the Primary Visual Cortex , 2007, Neuron.

[100]  G. Loewenstein,et al.  Decision making over time and under uncertainty: a common approach , 1991 .

[101]  P. Glimcher,et al.  Value Representations in the Primate Striatum during Matching Behavior , 2008, Neuron.

[102]  Ken-ichi Amemori,et al.  Localized Microstimulation of Primate Pregenual Cingulate Cortex Induces Negative Decision-Making , 2012, Nature Neuroscience.

[103]  C. Padoa-Schioppa,et al.  Neurons in the orbitofrontal cortex encode economic value , 2006, Nature.

[104]  D. Bernoulli Specimen theoriae novae de mensura sortis : translated into German and English , 1967 .

[105]  W. Schultz,et al.  Dopamine neurons of the monkey midbrain: contingencies of responses to stimuli eliciting immediate behavioral reactions. , 1990, Journal of neurophysiology.

[106]  E. Rowland Theory of Games and Economic Behavior , 1946, Nature.

[107]  M. D. Crutcher,et al.  Relations between movement and single cell discharge in the substantia nigra of the behaving monkey , 1983, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[108]  J. Brown,et al.  The electrophysiology of dopamine (D2) receptors: A study of the actions of dopamine on corticostriatal transmission , 1983, Neuroscience.

[109]  J. Bargas,et al.  D 1 Receptor Activation Enhances Evoked Discharge in Neostriatal Medium Spiny Neurons by Modulating an L-Type Ca 2 1 Conductance , 1997 .

[110]  Joseph J. Paton,et al.  The primate amygdala represents the positive and negative value of visual stimuli during learning , 2006, Nature.

[111]  M. Chesselet,et al.  Presynaptic regulation of neurotransmitter release in the brain: Facts and hypothesis , 1984, Neuroscience.

[112]  J. Mink THE BASAL GANGLIA: FOCUSED SELECTION AND INHIBITION OF COMPETING MOTOR PROGRAMS , 1996, Progress in Neurobiology.

[113]  A. Grace Phasic versus tonic dopamine release and the modulation of dopamine system responsivity: A hypothesis for the etiology of schizophrenia , 1991, Neuroscience.

[114]  Anatol C. Kreitzer,et al.  Distinct roles for direct and indirect pathway striatal neurons in reinforcement , 2012, Nature Neuroscience.

[115]  R. Duncan Luce,et al.  Individual Choice Behavior: A Theoretical Analysis , 1979 .

[116]  M. Fendt,et al.  Pain-relief learning in flies, rats, and man: basic research and applied perspectives , 2014, Learning & memory.

[117]  W. Schultz,et al.  The activity of pars compacta neurons of the monkey substantia nigra in relation to motor activation , 1983, Experimental Brain Research.

[118]  E. Macaluso,et al.  Stimulus-Driven Orienting of Visuo-Spatial Attention in Complex Dynamic Environments , 2011, Neuron.

[119]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[120]  J. Pearce,et al.  A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980, Psychological review.

[121]  G. Debreu,et al.  Cardinal utility for even-chance mixtures of pairs of sure prospects , 1959 .

[122]  C. Fiorillo,et al.  Optogenetic Mimicry of the Transient Activation of Dopamine Neurons by Natural Reward Is Sufficient for Operant Reinforcement , 2012, PloS one.

[123]  E. Ellinwood,et al.  Alterations in Baseline Activity and Quinpirole Sensitivity in Putative Dopamine Neurons in the Substantia Nigra and Ventral Tegmental Area after Withdrawal from Cocaine Pretreatment , 1998, Neuropsychopharmacology.

[124]  D. Ringach,et al.  Dynamics of Spatial Frequency Tuning in Macaque V1 , 2002, The Journal of Neuroscience.

[125]  T. Caraco,et al.  An empirical demonstration of risk-sensitive foraging preferences , 1980, Animal Behaviour.

[126]  榎本 一紀 Dopamine neurons learn to encode the long-term value of multiple future rewards , 2011 .

[127]  N. P. Bichot,et al.  Perceptual and motor processing stages identified in the activity of macaque frontal eye field neurons during visual search. , 1996, Journal of neurophysiology.

[128]  J. Glowinski,et al.  Effect of noxious tail pinch on the discharge rate of mesocortical and mesolimbic dopamine neurons: selective activation of the mesocortical system , 1989, Brain Research.

[129]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[130]  E. Vaadia,et al.  Midbrain dopamine neurons encode decisions for future action , 2006, Nature Neuroscience.

[131]  R. Wightman,et al.  Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens , 2007, Nature Neuroscience.

[132]  Ilana B. Witten,et al.  Recombinase-Driver Rat Lines: Tools, Techniques, and Optogenetic Application to Dopamine-Mediated Reinforcement , 2011, Neuron.

[133]  A. Bennett The Origin of Species by means of Natural Selection; or the Preservation of Favoured Races in the Struggle for Life , 1872, Nature.

[134]  J. Bentham An Introduction to the Principles of Morals and Legislation , 1945, Princeton Readings in Political Thought.

[135]  S. Floresco,et al.  Overriding Phasic Dopamine Signals Redirects Action Selection during Risk/Reward Decision Making , 2014, Neuron.

[136]  R. Wightman,et al.  Phasic Nucleus Accumbens Dopamine Encodes Risk-Based Decision-Making Behavior , 2012, Biological Psychiatry.

[137]  O. Hikosaka,et al.  Two types of dopamine neuron distinctly convey positive and negative motivational signals , 2009, Nature.

[138]  F. Guarraci,et al.  An electrophysiological characterization of ventral tegmental area dopaminergic neurons during differential pavlovian fear conditioning in the awake rabbit , 1999, Behavioural Brain Research.

[139]  P. Redgrave,et al.  Is the short-latency dopamine response too short to signal reward error? , 1999, Trends in Neurosciences.

[140]  P. Shepard,et al.  Lateral Habenula Stimulation Inhibits Rat Midbrain Dopamine Neurons through a GABAA Receptor-Mediated Mechanism , 2007, The Journal of Neuroscience.

[141]  K. Berridge,et al.  The neural basis of drug craving: An incentive-sensitization theory of addiction , 1993, Brain Research Reviews.

[142]  W. Pan,et al.  Dopamine Cells Respond to Predicted Events during Classical Conditioning: Evidence for Eligibility Traces in the Reward-Learning Network , 2005, The Journal of Neuroscience.

[143]  J. Kerr,et al.  Dopamine Receptor Activation Is Required for Corticostriatal Spike-Timing-Dependent Plasticity , 2008, The Journal of Neuroscience.

[144]  O. Hikosaka,et al.  Lateral habenula as a source of negative reward signals in dopamine neurons , 2007, Nature.

[145]  R. Rosenfeld Nature , 2009, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[146]  Ehsan Arabzadeh,et al.  Correlated physiological and perceptual effects of noise in a tactile stimulus , 2010, Proceedings of the National Academy of Sciences.

[147]  W. Schultz,et al.  Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm , 2003, The Journal of Neuroscience.

[148]  H. Markowitz The Utility of Wealth , 1952, Journal of Political Economy.

[149]  J. Hegdé,et al.  Temporal dynamics of shape analysis in macaque visual area V2. , 2004, Journal of neurophysiology.

[150]  Louis A. Chiodo,et al.  Sensory stimuli alter discharge rate of dopamine (DA) neurons: evidence for two functional types of DA cells in the substantia nigra , 1980, Brain Research.

[151]  Keiji Tanaka,et al.  Reward Association Affects Neuronal Responses to Visual Stimuli in Macaque TE and Perirhinal Cortices , 2006, The Journal of Neuroscience.

[152]  W. Schultz,et al.  Responses of monkey midbrain dopamine neurons during delayed alternation performance , 1991, Brain Research.

[153]  Isidore Gormezano,et al.  Effects of water deprivation on classical appetitive conditioning of the rabbit's jaw movement response , 1970 .

[154]  C. Fiorillo Two Dimensions of Value: Dopamine Neurons Represent Reward But Not Aversiveness , 2013, Science.

[155]  Minryung R. Song,et al.  Multiphasic Temporal Dynamics in Responses of Midbrain Dopamine Neurons to Appetitive and Aversive Stimuli , 2013, The Journal of Neuroscience.

[156]  Tomaso Poggio,et al.  Fast Readout of Object Identity from Macaque Inferior Temporal Cortex , 2005, Science.

[157]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[158]  R. Romo,et al.  Dopamine neurons code subjective sensory experience and uncertainty of perceptual decisions , 2011, Proceedings of the National Academy of Sciences.

[159]  A. Graybiel,et al.  Effect of the nigrostriatal dopamine system on acquired neural responses in the striatum of behaving monkeys. , 1994, Science.

[160]  A. Graybiel,et al.  Prolonged Dopamine Signalling in Striatum Signals Proximity and Value of Distant Rewards , 2013, Nature.

[161]  Sabrina Ravel,et al.  Responses of Tonically Active Neurons in the Monkey Striatum Discriminate between Motivationally Opposing Stimuli , 2003, The Journal of Neuroscience.

[162]  M. Machina Choice under Uncertainty: Problems Solved and Unsolved , 1987 .

[163]  K. Deisseroth,et al.  Optogenetic Interrogation of Dopaminergic Modulation of the Multiple Phases of Reward-Seeking Behavior , 2011, The Journal of Neuroscience.

[164]  Jeannette A. M. Lorteije,et al.  The Formation of Hierarchical Decisions in the Visual Cortex , 2015, Neuron.

[165]  Masayuki Matsumoto,et al.  Distinct Representations of Cognitive and Motivational Signals in Midbrain Dopamine Neurons , 2013, Neuron.

[166]  Timothy E. J. Behrens,et al.  Double dissociation of value computations in orbitofrontal and anterior cingulate neurons , 2011, Nature Neuroscience.

[167]  A. Grace,et al.  Regulation of firing of dopaminergic neurons and control of goal-directed behaviors , 2007, Trends in Neurosciences.

[168]  W. Schultz,et al.  Responses of rat pallidum cells to cortex stimulation and effects of altered dopaminergic activity , 1985, Neuroscience.

[169]  H. de Wit,et al.  Determination of discount functions in rats with an adjusting-amount procedure. , 1997, Journal of the experimental analysis of behavior.

[170]  D. Bernoulli Exposition of a New Theory on the Measurement of Risk , 1954 .

[171]  W. Schultz,et al.  Dopamine neurons of the monkey midbrain: contingencies of responses to active touch during self-initiated arm movements. , 1990, Journal of neurophysiology.

[172]  G. Chapman,et al.  Playing for peanuts: Why is risk seeking more common for low-stakes gambles? , 2005 .

[173]  W. Schultz,et al.  Dopamine responses comply with basic assumptions of formal learning theory , 2001, Nature.

[174]  Anna E. Ipata,et al.  Neurons in the lateral intraparietal area create a priority map by the combination of disparate signals , 2008, Experimental Brain Research.

[175]  Kenji Kawano,et al.  Global and fine information coded by single neurons in the temporal visual cortex , 1999, Nature.

[176]  C. Pennartz,et al.  A unified selection signal for attention and reward in primary visual cortex , 2013, Proceedings of the National Academy of Sciences.

[177]  T. Robinson,et al.  The role of dopamine in the accumbens core in the expression of Pavlovian‐conditioned responses , 2012, The European journal of neuroscience.

[178]  Nao Chuhma,et al.  Dopamine Neurons Control Striatal Cholinergic Neurons via Regionally Heterogeneous Dopamine and Glutamate Signaling , 2014, Neuron.

[179]  K. Doya,et al.  Validation of Decision-Making Models and Analysis of Decision Variables in the Rat Basal Ganglia , 2009, The Journal of Neuroscience.

[180]  S. Nicola,et al.  Basolateral Amygdala Neurons Facilitate Reward-Seeking Behavior by Exciting Nucleus Accumbens Neurons , 2008, Neuron.

[181]  W. Schultz,et al.  Coding of Reward Risk by Orbitofrontal Neurons Is Mostly Distinct from Coding of Reward Value , 2010, Neuron.

[182]  D. Robinson,et al.  Behavioral enhancement of visual responses in monkey cerebral cortex. I. Modulation in posterior parietal cortex related to selective visual attention. , 1981, Journal of neurophysiology.

[183]  G. Bi,et al.  Gain in sensitivity and loss in temporal contrast of STDP by dopaminergic modulation at hippocampal synapses , 2009, Proceedings of the National Academy of Sciences.

[184]  J. Roeper Dissecting the diversity of midbrain dopamine neurons , 2013, Trends in Neurosciences.

[185]  W. Newsome,et al.  The temporal precision of reward prediction in dopamine neurons , 2008, Nature Neuroscience.

[186]  A. Grace,et al.  Afferent modulation of dopamine neuron firing differentially regulates tonic and phasic dopamine transmission , 2003, Nature Neuroscience.

[187]  Jung Hoon Sul,et al.  Role of Striatum in Updating Values of Chosen Actions , 2009, The Journal of Neuroscience.

[188]  L. Chelazzi,et al.  Altering Spatial Priority Maps via Reward-Based Learning , 2014, The Journal of Neuroscience.