History- and current instruction-based coding of forthcoming behavioral outcomes in the striatum.

Animals optimize behaviors by predicting future critical events based on histories of actions and their outcomes. When behavioral outcomes like reward and aversion are signaled by current external cues, actions are directed to acquire the reward and avoid the aversion. The basal ganglia are thought to be the brain locus for reward-based adaptive action planning and learning. To understand the role of striatum in coding outcomes of forthcoming behavioral responses, we addressed two specific questions. First, how are the histories of reward and aversion used for encoding forthcoming outcomes in the striatum during a series of instructed behavioral responses? Second, how are the behavioral responses and their instructed outcomes represented in the striatum? We recorded discharges of 163 presumed projection neurons in the striatum while monkeys performed a visually instructed lever-release task for reward, aversion, and sound outcomes, whose occurrences could be estimated by their histories. Before outcome instruction, discharge rates of a subset of neurons activated in this epoch showed positive or negative regression slopes with reward history (24/44), that is, to the number of trials since the last reward trial, which changed in parallel with reward probability of current trials. The history effect was also observed for the aversion outcome but in far fewer neurons (3/44). Once outcomes were instructed in the same task, neurons selectively encoded the outcomes before and after behavioral responses (reward, 46/70; aversion, 6/70; sound, 6/70). The history- and current instruction-based coding of forthcoming behavioral outcomes in the striatum might underlie outcome-oriented behavioral modulation.

[1]  E. Thorndike Animal intelligence : an experimental study of the associative processes in animals / by Edward L. Thorndike. , .

[2]  C D Marsden,et al.  Which motor disorder in Parkinson's disease indicates the true motor function of the basal ganglia? , 2008, Ciba Foundation symposium.

[3]  Y. Miyashita,et al.  Neural organization for the long-term memory of paired associates , 1991, Nature.

[4]  B. Balleine,et al.  Motivational control of goal-directed action , 1994 .

[5]  Joel L. Davis,et al.  A Model of How the Basal Ganglia Generate and Use Neural Signals That Predict Reinforcement , 1994 .

[6]  A M Graybiel,et al.  The basal ganglia and adaptive motor control. , 1994, Science.

[7]  A. Graybiel,et al.  Temporal and spatial characteristics of tonically active neurons of the primate's striatum. , 1995, Journal of neurophysiology.

[8]  K Watanabe,et al.  Neural information transferred from the putamen to the globus pallidus during learned movement in the monkey. , 1996, Journal of neurophysiology.

[9]  C. Marsden,et al.  Learning manual pursuit tracking skills in patients with Parkinson's disease. , 1997, Brain : a journal of neurology.

[10]  B. Richmond,et al.  Neuronal Signals in the Monkey Ventral Striatum Related to Progress through a Predictable Series of Trials , 1998, The Journal of Neuroscience.

[11]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[12]  J. Hollerman,et al.  Influence of reward expectation on behavior-related neuronal activity in primate striatum. , 1998, Journal of neurophysiology.

[13]  O. Hikosaka,et al.  Expectation of reward modulates cognitive signals in the basal ganglia , 1998, Nature Neuroscience.

[14]  W. Schultz,et al.  Relative reward preference in primate orbitofrontal cortex , 1999, Nature.

[15]  E. Miller,et al.  Prospective Coding for Objects in Primate Prefrontal Cortex , 1999, The Journal of Neuroscience.

[16]  K. Doya Complementary roles of basal ganglia and cerebellum in learning and motor control , 2000, Current Opinion in Neurobiology.

[17]  W. Schultz,et al.  Influence of expectation of different rewards on behavior-related neuronal activity in the striatum. , 2001, Journal of neurophysiology.

[18]  T. Ono,et al.  Retrospective and prospective coding for predicted reward in the sensory thalamus , 2001, Nature.

[19]  Wolfram Schultz,et al.  Behavioral reactions reflecting differential reward expectations in monkeys , 2001, Experimental Brain Research.

[20]  Alan Grafen,et al.  Modern statistics for the life sciences , 2002 .

[21]  M. Walton,et al.  The Role of Rat Medial Frontal Cortex in Effort-Based Decision Making , 2002, The Journal of Neuroscience.

[22]  A. Graybiel,et al.  A Network Representation of Response Probability in the Striatum , 2002, Neuron.

[23]  J. Tanji,et al.  Numerical representation for action in the parietal cortex of the monkey , 2002, Nature.

[24]  B. Richmond,et al.  Anterior Cingulate: Single Neuronal Signals Related to Degree of Reward Expectancy , 2002, Science.

[25]  Sabrina Ravel,et al.  Responses of Tonically Active Neurons in the Monkey Striatum Discriminate between Motivationally Opposing Stimuli , 2003, The Journal of Neuroscience.

[26]  Matthew F S Rushworth,et al.  Functional Specialization within Medial Frontal Cortex of the Anterior Cingulate for Evaluating Effort-Related Decisions , 2003, The Journal of Neuroscience.

[27]  Wolfram Schultz,et al.  Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. , 2003, Journal of neurophysiology.

[28]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[29]  G. Schoenbaum,et al.  Neural Encoding in Ventral Striatum during Olfactory Discrimination Learning , 2003, Neuron.

[30]  Tatsuo K Sato,et al.  Correlated Coding of Motivation and Outcome of Decision by Dopamine Neurons , 2003, The Journal of Neuroscience.

[31]  O. Hikosaka,et al.  Dopamine Neurons Can Represent Context-Dependent Prediction Error , 2004, Neuron.

[32]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[33]  Masataka Watanabe,et al.  Long‐ and short‐range reward expectancy in the primate orbitofrontal cortex , 2004, The European journal of neuroscience.

[34]  E. Vaadia,et al.  Coincident but Distinct Messages of Midbrain Dopamine and Striatal Tonically Active Neurons , 2004, Neuron.

[35]  A. Nieder Counting on neurons: the neurobiology of numerical competence , 2005, Nature Reviews Neuroscience.

[36]  A. Graybiel,et al.  Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories , 2005, Nature.

[37]  K. Doya,et al.  Representation of Action-Specific Reward Values in the Striatum , 2005, Science.

[38]  山田 洋 Tonically active neurons in the primate caudate nucleus and putamen differentially encode instructed motivational outcomes of action , 2005 .

[39]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[40]  C. Padoa-Schioppa,et al.  Neurons in the orbitofrontal cortex encode economic value , 2006, Nature.

[41]  H. Yin,et al.  The role of the basal ganglia in habit formation , 2006, Nature Reviews Neuroscience.

[42]  W. Schultz,et al.  Influences of Rewarding and Aversive Outcomes on Activity in Macaque Lateral Prefrontal Cortex , 2006, Neuron.

[43]  Shintaro Funahashi,et al.  Reward-period Activity in Primate Dorsolateral Prefrontal and Orbitofrontal Neurons Is Affected by Reward Schedules , 2006, Journal of Cognitive Neuroscience.

[44]  Masato Inoue,et al.  Neurons in the macaque orbitofrontal cortex code relative preference of both rewarding and aversive outcomes , 2007, Neuroscience Research.

[45]  Barry J Richmond,et al.  Dynamic changes in representations of preceding and upcoming reward in monkey orbitofrontal cortex. , 2008, Cerebral cortex.

[46]  Richard S. Sutton,et al.  Reinforcement Learning , 1992, Handbook of Machine Learning.

[47]  K. Campbell,et al.  A neural correlate of response bias in monkey caudate nucleus , 2022 .