Encoding of action history in the rat ventral striatum.

In a dynamic environment, animals need to update information about the rewards expected from their alternative actions continually to make optimal choices for its survival. Because the reward resulting from a given action can be substantially delayed, the process of linking a reward to its causative action would be facilitated by memory signals related to the animal's previous actions. Although the ventral striatum has been proposed to play a key role in updating the information about the rewards expected from specific actions, it is not known whether the signals related to previous actions exist in the ventral striatum. In the present study, we recorded neuronal ensemble activity in the rat ventral striatum during a visual discrimination task and investigated whether neuronal activity in the ventral striatum encoded signals related to animal's previous actions. The results show that many neurons modulated their activity according to the animal's goal choice in the previous trial, indicating that memory signals for previous actions are available in the ventral striatum. In contrast, few neurons conveyed signals on impending goal choice of the animal, suggesting the absence of decision signals in the ventral striatum. Memory signals for previous actions might contribute to the process of updating the estimates of rewards expected from alternative actions in the ventral striatum.

[1]  M. Jung,et al.  Dynamics of Population Code for Working Memory in the Prefrontal Cortex , 2003, Neuron.

[2]  Okihide Hikosaka,et al.  Functional differences between macaque prefrontal cortex and caudate nucleus during eye movements with and without reward , 2006, Experimental Brain Research.

[3]  Joel L. Davis,et al.  A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement , 1994 .

[4]  H. Seo,et al.  Dynamic signals related to choices and outcomes in the dorsolateral prefrontal cortex. , 2007, Cerebral cortex.

[5]  E. Lynd-Balta,et al.  The orbital and medial prefrontal circuit through the primate basal ganglia , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[6]  J. O'Doherty,et al.  Reward representations and reward-related learning in the human brain: insights from neuroimaging , 2004, Current Opinion in Neurobiology.

[7]  S. Wiener,et al.  Position sensitivity in phasically discharging nucleus accumbens neurons of rats alternating between tasks requiring complementary types of spatial cues , 2001, Neuroscience.

[8]  H. Groenewegen,et al.  Integration and segregation of limbic cortico-striatal loops at the thalamic level: an experimental tracing study in rats , 1999, Journal of Chemical Neuroanatomy.

[9]  A. Lavoie,et al.  Spatial, movement- and reward-sensitive discharge by medial ventral striatum neurons of rats , 1994, Brain Research.

[10]  W. Schultz.  Behavioral theories and the neurophysiology of reward. , 2006, Annual review of psychology.

[11]  Garrett E. Alexander.  Basal ganglia , 1998 .

[12]  P. Janak,et al.  Mesolimbic Neuronal Activity across Behavioral States , 1999, Annals of the New York Academy of Sciences.

[13]  W. Schultz,et al.  Neuronal activity in monkey ventral striatum related to the expectation of reward , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  Wolfram Schultz,et al.  Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. , 2003, Journal of neurophysiology.

[15]  Jeffrey C. Cooper,et al.  Functional magnetic resonance imaging of reward prediction , 2005, Current opinion in neurology.

[16]  G. Schoenbaum,et al.  Neural Encoding in Ventral Striatum during Olfactory Discrimination Learning , 2003, Neuron.

[17]  S. Haber,et al.  Reward-Related Cortical Inputs Define a Large Striatal Region in Primates That Interface with Associative Cortical Connections, Providing a Substrate for Incentive-Based Learning , 2006, The Journal of Neuroscience.

[18]  W. Pan,et al.  Dopamine Cells Respond to Predicted Events during Classical Conditioning: Evidence for Eligibility Traces in the Reward-Learning Network , 2005, The Journal of Neuroscience.

[19]  C. Pennartz,et al.  The nucleus accumbens as a complex of functionally distinct neuronal ensembles: An integration of behavioural, electrophysiological and anatomical data , 1994, Progress in Neurobiology.

[20]  Masataka Watanabe.  Reward expectancy in primate prefrental neurons , 1996, Nature.

[21]  G. Shepherd.  The Synaptic Organization of the Brain , 1979 .

[22]  L. Chen,et al.  Neuronal responses in the frontal cortico-basal ganglia system during delayed matching-to-sample task: ensemble recording in freely moving rats , 2001, Experimental Brain Research.

[23]  D. Barraclough,et al.  Prefrontal cortex and decision making in a mixed-strategy game , 2004, Nature Neuroscience.

[24]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[25]  Jeong‐Wook Ghim,et al.  Learning-Induced Enduring Changes in Functional Connectivity among Prefrontal Cortical Neurons , 2007, The Journal of Neuroscience.

[26]  P. Redgrave,et al.  The basal ganglia: a vertebrate solution to the selection problem? , 1999, Neuroscience.

[27]  O. Hikosaka,et al.  Functional properties of monkey caudate neurons. I. Activities related to saccadic eye movements. , 1989, Journal of neurophysiology.

[28]  K. Doya,et al.  Representation of Action-Specific Reward Values in the Striatum , 2005, Science.

[29]  H. Fields,et al.  Inhibitions of Nucleus Accumbens Neurons Encode a Gating Signal for Reward-Directed Behavior , 2006, The Journal of Neuroscience.

[30]  O. Hikosaka,et al.  Expectation of reward modulates cognitive signals in the basal ganglia , 1998, Nature Neuroscience.

[31]  C. Cavada,et al.  The anatomical connections of the macaque monkey orbitofrontal cortex. A review. , 2000, Cerebral cortex.

[32]  K. Doya,et al.  The computational neurobiology of learning and reward , 2006, Current Opinion in Neurobiology.

[33]  Joel L. Davis,et al.  Adaptive Critics and the Basal Ganglia , 1995 .

[34]  W. Schultz,et al.  A neural network model with dopamine-like reinforcement signal that learns a spatial delayed response task , 1999, Neuroscience.

[35]  E. Miller,et al.  Different time courses of learning-related activity in the prefrontal cortex and striatum , 2005, Nature.

[36]  Bruce L McNaughton,et al.  Apparent Encoding of Sequential Context in Rat Medial Prefrontal Cortex Is Accounted for by Behavioral Variability , 2006, The Journal of Neuroscience.

[37]  S. Nicola.  The nucleus accumbens as part of a basal ganglia action selection circuit , 2007, Psychopharmacology.

[38]  Nikolaus R. McFarland,et al.  The Concept of the Ventral Striatum in Nonhuman Primates , 1999, Annals of the New York Academy of Sciences.

[39]  E. Bézard,et al.  Shaping of Motor Responses by Incentive Values through the Basal Ganglia , 2007, The Journal of Neuroscience.

[40]  Richard S. Sutton,et al.  Time-Derivative Models of Pavlovian Reinforcement , 1990 .

[41]  M. Jung,et al.  Fast spiking and regular spiking neural correlates of fear conditioning in the medial prefrontal cortex of the rat. , 2001, Cerebral cortex.

[42]  M. Shadlen,et al.  Effect of Expected Reward Magnitude on the Response of Neurons in the Dorsolateral Prefrontal Cortex of the Macaque , 1999, Neuron.

[43]  Daeyeol Lee,et al.  Functional Specialization of the Primate Frontal Cortex during Decision Making , 2007, The Journal of Neuroscience.

[44]  B. Everitt,et al.  Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex , 2002, Neuroscience & Biobehavioral Reviews.

[45]  Daeyeol Lee.  Neural basis of quasi-rational decision making , 2006, Current Opinion in Neurobiology.

[46]  N. Daw,et al.  Reinforcement learning models of the dopamine system and their behavioral implications , 2003 .

[47]  M. Gabriel,et al.  Learning and Computational Neuroscience: Foundations of Adaptive Networks , 1990 .

[48]  S. Wiener,et al.  Neurons in hippocampal afferent zones of rat striatum parse routes into multi‐pace segments during maze navigation , 2004, The European journal of neuroscience.

[49]  R. Vertes.  Differential projections of the infralimbic and prelimbic cortex in the rat , 2004, Synapse.

[50]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[51]  Howard L Fields,et al.  Cue-evoked firing of nucleus accumbens neurons encodes motivational significance during a discriminative stimulus task. , 2004, Journal of neurophysiology.

[52]  G. E. Alexander,et al.  Preparation for movement: neural representations of intended direction in three motor areas of the monkey. , 1990, Journal of neurophysiology.

[53]  Young Ho Kim,et al.  Role of active movement in place‐specific firing of hippocampal neurons , 2005, Hippocampus.