Nucleus accumbens neurons encode predicted and ongoing reward costs in rats

Efficient decision‐making requires that animals consider both the benefits and the costs of potential actions, such as the amount of effort or temporal delay involved in reward seeking. The nucleus accumbens (NAc) has been implicated in the ability to choose between options with different costs and overcome high costs when necessary, but it is not clear how NAc processing contributes to this role. Here, neuronal activity in the rat NAc was monitored using multi‐neuron electrophysiology during two cost‐based decision tasks in which either reward effort or reward delay was manipulated. In each task, distinct visual cues predicted high‐value (low effort/immediate) and low‐value (high effort/delayed) rewards. After training, animals exhibited a behavioral preference for high‐value rewards, yet overcame high costs when necessary to obtain rewards. Electrophysiological analysis indicated that a subgroup of NAc neurons exhibited phasic increases in firing rate during cue presentations. In the effort‐based decision task (but not the delay‐based task), this population reflected the cost‐discounted value of the future response. In contrast, other subgroups of cells were activated during response initiation or reward delivery, but activity did not differ on the basis of reward cost. Finally, another population of cells exhibited sustained changes in firing rate while animals completed high‐effort requirements or waited for delayed rewards. These findings are consistent with previous reports that implicate NAc function in reward prediction and behavioral allocation during reward‐seeking behavior, and suggest a mechanism by which NAc activity contributes to both cost‐based decisions and actual cost expenditure.

[1]  Howard L Fields,et al.  Cue‐evoked encoding of movement planning and execution in the rat nucleus accumbens , 2007, The Journal of physiology.

[2]  J. Salamone,et al.  Nucleus accumbens dopamine depletions make rats more sensitive to high ratio requirements but do not impair primary food reinforcement , 1999, Neuroscience.

[3]  Robert E. Hampson,et al.  Firing patterns of nucleus accumbens neurons during cocaine self-administration in rats , 1993, Brain Research.

[4]  Joseph J. Paton,et al.  Expectation Modulates Neural Responses to Pleasant and Aversive Stimuli in Primate Amygdala , 2007, Neuron.

[5]  S. Nicola,et al.  Firing of nucleus accumbens neurons during the consummatory phase of a discriminative stimulus task depends on previous reward predictive cues. , 2004, Journal of neurophysiology.

[6]  C. Pennartz,et al.  The nucleus accumbens as a complex of functionally distinct neuronal ensembles: An integration of behavioural, electrophysiological and anatomical data , 1994, Progress in Neurobiology.

[7]  Howard L Fields,et al.  Cue-evoked firing of nucleus accumbens neurons encodes motivational significance during a discriminative stimulus task. , 2004, Journal of neurophysiology.

[8]  H. Fields,et al.  Inhibitions of Nucleus Accumbens Neurons Encode a Gating Signal for Reward-Directed Behavior , 2006, The Journal of Neuroscience.

[9]  M. Roitman,et al.  Nucleus accumbens neurons encode Pavlovian approach behaviors: evidence from an autoshaping paradigm , 2006, The European journal of neuroscience.

[10]  Wolfram Schultz,et al.  Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. , 2003, Journal of neurophysiology.

[11]  J. Salamone,et al.  Ratio and time requirements on operant schedules: effort‐related effects of nucleus accumbens dopamine depletions , 2005, The European journal of neuroscience.

[12]  D. Woodward,et al.  Electrophysiological and pharmacological evidence for the role of the nucleus accumbens in cocaine self-administration in freely moving rats , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[13]  J. Salamone,et al.  Motivational views of reinforcement: implications for understanding the behavioral functions of nucleus accumbens dopamine , 2002, Behavioural Brain Research.

[14]  Kay M. Tye,et al.  Rapid strengthening of thalamo-amygdala synapses mediates cue–reward learning , 2008, Nature.

[15]  S. Nicola The nucleus accumbens as part of a basal ganglia action selection circuit , 2007, Psychopharmacology.

[16]  Benjamin M. Robinson,et al.  Delay discounting in schizophrenia , 2007, Cognitive neuropsychiatry.

[17]  P. Glimcher,et al.  The neural correlates of subjective value during intertemporal choice , 2007, Nature Neuroscience.

[18]  M. Walton,et al.  Dissociable cost and benefit encoding of future rewards by mesolimbic dopamine , 2009, Nature Neuroscience.

[19]  R. Carelli,et al.  Evidence That Separate Neural Circuits in the Nucleus Accumbens Encode Cocaine Versus “Natural” (Water and Food) Reward , 2000, The Journal of Neuroscience.

[20]  S A Deadwyler,et al.  A comparison of nucleus accumbens neuronal firing patterns during cocaine self-administration and water reinforcement in rats , 1994, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[21]  S. Nicola,et al.  Dissociation of the role of nucleus accumbens dopamine in responding to reward-predictive cues and waiting for reward , 2004, Behavioural Brain Research.

[22]  Howard L Fields,et al.  Encoding of Palatability and Appetitive Behaviors by Distinct Neuronal Populations in the Nucleus Accumbens , 2005, The Journal of Neuroscience.

[23]  G. Schoenbaum,et al.  Neural Encoding in Ventral Striatum during Olfactory Discrimination Learning , 2003, Neuron.

[24]  M. Walton,et al.  The Role of Rat Medial Frontal Cortex in Effort-Based Decision Making , 2002, The Journal of Neuroscience.

[25]  J. Salamone,et al.  D1 or D2 antagonism in nucleus accumbens core or dorsomedial shell suppresses lever pressing for food but leads to compensatory increases in chow consumption , 2001, Pharmacology Biochemistry and Behavior.

[26]  A. Scheres,et al.  Temporal Reward Discounting in Attention-Deficit/Hyperactivity Disorder: The Contribution of Symptom Domains, Reward Magnitude, and Session Length , 2010, Biological Psychiatry.

[27]  J. Salamone,et al.  Effects of Dopamine Antagonists and Accumbens Dopamine Depletions on Time-Constrained Progressive-Ratio Performance , 1998, Pharmacology Biochemistry and Behavior.

[28]  Joshua L. Jones,et al.  Phasic Nucleus Accumbens Dopamine Release Encodes Effort- and Delay-Related Costs , 2010, Biological Psychiatry.

[29]  Rudolf N. Cardinal,et al.  Neural systems implicated in delayed and probabilistic reinforcement , 2006, Neural Networks.

[30]  Trevor W Robbins,et al.  Interactions between Serotonin and Dopamine in the Control of Impulsive Choice in Rats: Therapeutic Implications for Impulse Control Disorders , 2005, Neuropsychopharmacology.

[31]  W. Schultz,et al.  Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. , 2001, Journal of neurophysiology.

[32]  J. Deakin,et al.  Effects of quinolinic acid-induced lesions of the orbital prefrontal cortex on inter-temporal choice: a quantitative analysis , 2002, Psychopharmacology.

[33]  Masataka Watanabe Reward expectancy in primate prefrental neurons , 1996, Nature.

[34]  L. Peoples,et al.  Firing patterns of accumbal neurons during a pavlovian-conditioned approach task. , 2006, Journal of neurophysiology.

[35]  Wolfgang Hauber,et al.  Prefrontostriatal circuitry regulates effort-related decision making. , 2009, Cerebral cortex.

[36]  S. Nicola,et al.  Basolateral Amygdala Neurons Facilitate Reward-Seeking Behavior by Exciting Nucleus Accumbens Neurons , 2008, Neuron.

[37]  Mark O West,et al.  Differences between accumbens core and shell neurons exhibiting phasic firing patterns related to drug-seeking behavior during a discriminative-stimulus task. , 2004, Journal of neurophysiology.

[38]  R. Carelli,et al.  Abstinence from Cocaine Self-Administration Heightens Neural Encoding of Goal-Directed Behaviors in the Accumbens , 2005, Neuropsychopharmacology.

[39]  G. Paxinos,et al.  The Rat Brain in Stereotaxic Coordinates , 1983 .

[40]  E. Bowman,et al.  Rat nucleus accumbens neurons predominantly respond to the outcome-related properties of conditioned stimuli rather than their behavioral-switching properties. , 2005, Journal of neurophysiology.

[41]  J. Salamone,et al.  Effort-related functions of nucleus accumbens dopamine and associated forebrain circuits , 2007, Psychopharmacology.

[42]  Charles J. Wilson,et al.  Connectivity and Convergence of Single Corticostriatal Axons , 1998, The Journal of Neuroscience.

[43]  M. Walton,et al.  Comparing the role of the anterior cingulate cortex and 6‐hydroxydopamine nucleus accumbens lesions on operant effort‐based decision making , 2009, The European journal of neuroscience.

[44]  M. Roesch,et al.  Ventral Striatal Neurons Encode the Value of the Chosen Action in Rats Deciding between Differently Delayed or Sized Rewards , 2009, The Journal of Neuroscience.

[45]  K. Doya Modulators of decision making , 2008, Nature Neuroscience.

[46]  Y. Lacasse,et al.  From the authors , 2005, European Respiratory Journal.

[47]  P. W. German,et al.  A Pause in Nucleus Accumbens Neuron Firing Is Required to Initiate and Maintain Feeding , 2010, The Journal of Neuroscience.

[48]  R. Carelli The nucleus accumbens and reward: neurophysiological investigations in behaving animals. , 2002, Behavioral and cognitive neuroscience reviews.

[49]  J. Hollerman,et al.  Influence of reward expectation on behavior-related neuronal activity in primate striatum. , 1998, Journal of neurophysiology.

[50]  T. Robbins,et al.  Impulsive Choice Induced in Rats by Lesions of the Nucleus Accumbens Core , 2001, Science.

[51]  K. Doya,et al.  Representation of Action-Specific Reward Values in the Striatum , 2005, Science.

[52]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[53]  L. Green,et al.  A discounting framework for choice with delayed and probabilistic rewards. , 2004, Psychological bulletin.

[54]  M. West,et al.  Persistent Cue-Evoked Activity of Accumbens Neurons after Prolonged Abstinence from Self-Administered Cocaine , 2003, The Journal of Neuroscience.

[55]  S. Floresco,et al.  Dopaminergic and Glutamatergic Regulation of Effort- and Delay-Based Decision Making , 2008, Neuropsychopharmacology.

[56]  J. Deakin,et al.  Effect of quinolinic acid-induced lesions of the nucleus accumbens core on performance on a progressive ratio schedule of reinforcement: implications for inter-temporal choice , 2008, Psychopharmacology.

[57]  C. M. Bradshaw,et al.  Effects of quinolinic acid-induced lesions of the nucleus accumbens core on inter-temporal choice: a quantitative analysis , 2007, Psychopharmacology.

[58]  Timothy Edward John Behrens,et al.  Effort-Based Cost–Benefit Valuation and the Human Brain , 2009, The Journal of Neuroscience.

[59]  Joshua L. Jones,et al.  Behavioral and Electrophysiological Indices of Negative Affect Predict Cocaine Self-Administration , 2008, Neuron.

[60]  M. Walton,et al.  Separate neural pathways process different decision costs , 2006, Nature Neuroscience.

[61]  D. S. Zahm,et al.  An integrative neuroanatomical perspective on some subcortical substrates of adaptive responding with emphasis on the nucleus accumbens , 2000, Neuroscience & Biobehavioral Reviews.

[62]  M. Roitman,et al.  Nucleus Accumbens Neurons Are Innately Tuned for Rewarding and Aversive Taste Stimuli, Encode Their Predictors, and Are Linked to Motor Output , 2005, Neuron.

[63]  R. Carelli Nucleus accumbens cell firing during goal-directed behaviors for cocaine vs. ‘natural’ reinforcement , 2002, Physiology & Behavior.

[64]  D. S. Zahm,et al.  Functional‐anatomical Implications of the Nucleus Accumbens Core and Shell Subterritories , 1999, Annals of the New York Academy of Sciences.

[65]  Wolfram Schultz,et al.  Relative reward processing in primate striatum , 2005, Experimental Brain Research.

[66]  E. Rolls,et al.  Hunger and satiety modify the responses of olfactory and visual neurons in the primate orbitofrontal cortex. , 1996, Journal of neurophysiology.

[67]  S. Floresco,et al.  Amygdala-prefrontal cortical circuitry regulates effort-based decision making. , 2006, Cerebral cortex.

[68]  Timothy E. J. Behrens,et al.  Choice, uncertainty and value in prefrontal and cingulate cortex , 2008, Nature Neuroscience.

[69]  J. Salamone,et al.  Anhedonia or anergia? Effects of haloperidol and nucleus accumbens dopamine depletion on instrumental response selection in a T-maze cost/benefit procedure , 1994, Behavioural Brain Research.

[70]  M. Nicolelis,et al.  Reconstructing the Engram: Simultaneous, Multisite, Many Single Neuron Recordings , 1997, Neuron.