Habit formation coincides with shifts in reinforcement representations in the sensorimotor striatum.

Evaluating outcomes of behavior is a central function of the striatum. In circuits engaging the dorsomedial striatum, sensitivity to goal value is accentuated during learning, whereas outcome sensitivity is thought to be minimal in the dorsolateral striatum and its habit-related corticostriatal circuits. However, a distinct population of projection neurons in the dorsolateral striatum exhibits selective sensitivity to rewards. Here, we evaluated the outcome-related signaling in such neurons as rats performed an instructional T-maze task for two rewards. As the rats formed maze-running habits and then changed behavior after reward devaluation, we detected outcome-related spike activity in 116 units out of 1,479 recorded units. During initial training, nearly equal numbers of these units fired preferentially either after rewarded runs or after unrewarded runs, and the majority were responsive at only one of two reward locations. With overtraining, as habits formed, firing in nonrewarded trials almost disappeared, and reward-specific firing declined. Thus error-related signaling was lost, and reward signaling became generalized. Following reward devaluation, in an extinction test, postgoal activity was nearly undetectable, despite accurate running. Strikingly, when rewards were then returned, postgoal activity reappeared and recapitulated the original early response pattern, with nearly equal numbers responding to rewarded and unrewarded runs and to single rewards. These findings demonstrate that outcome evaluation in the dorsolateral striatum is highly plastic and tracks stages of behavioral exploration and exploitation. These signals could be a new target for understanding compulsive behaviors that involve changes to dorsal striatum function.

[1]  T. Robbins,et al.  Drug Addiction: Updating Actions to Habits to Compulsions Ten Years On. , 2016, Annual review of psychology.

[2]  A. Redish,et al.  Hippocampus and subregions of the dorsal striatum respond differently to a behavioral strategy change on a spatial navigation task. , 2015, Journal of neurophysiology.

[3]  Theresa M. Desrochers,et al.  Habit Learning by Naive Macaques Is Marked by Response Sharpening of Striatal Neurons Representing the Cost and Outcome of Acquired Action Sequences , 2015, Neuron.

[4]  B. Balleine,et al.  Plasticity in striatopallidal projection neurons mediates the acquisition of habitual actions , 2015, The European journal of neuroscience.

[5]  P. Rueda-Orozco,et al.  The striatum multiplexes contextual and kinematic information to constrain motor habits execution , 2014, Nature Neuroscience.

[6]  P. Janak,et al.  Habitual responding for alcohol depends upon both AMPA and D2 receptor signaling in the dorsolateral striatum , 2014, Front. Behav. Neurosci..

[7]  A. Graybiel,et al.  Neurons in the Ventral Striatum Exhibit Cell-Type-Specific Representations of Outcome during Learning , 2014, Neuron.

[8]  A. Graybiel,et al.  Differential Entrainment and Learning-Related Dynamics of Spike and Local Field Potential Activity in the Sensorimotor and Associative Striatum , 2014, The Journal of Neuroscience.

[9]  Kyle S. Smith,et al.  Investigating habits: strategies, technologies and models , 2014, Front. Behav. Neurosci..

[10]  Xin Jin,et al.  Basal Ganglia Subcircuits Distinctively Encode the Parsing and Concatenation of Action Sequences , 2014, Nature Neuroscience.

[11]  P. Dayan,et al.  Goals and Habits in the Brain , 2013, Neuron.

[12]  R. Costa,et al.  Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions , 2013, Nature Communications.

[13]  A. Graybiel,et al.  Prolonged Dopamine Signalling in Striatum Signals Proximity and Value of Distant Rewards , 2013, Nature.

[14]  Kyle S. Smith,et al.  A Dual Operator View of Habitual Behavior Reflecting Cortical and Striatal Dynamics , 2013, Neuron.

[15]  L. Saksida,et al.  GluN2B in corticostriatal circuits governs choice learning and choice shifting , 2013, Nature Neuroscience.

[16]  Kyle S. Smith,et al.  Using optogenetics to study habits , 2013, Brain Research.

[17]  B. Everitt,et al.  Hierarchical recruitment of phasic dopamine signaling in the striatum during the progression of cocaine use , 2012, Proceedings of the National Academy of Sciences.

[18]  Kyle S. Smith,et al.  Reversible online control of habitual behavior by optogenetic perturbation of medial prefrontal cortex , 2012, Proceedings of the National Academy of Sciences.

[19]  E. Simpson Faculty Opinions recommendation of Distinct roles for direct and indirect pathway striatal neurons in reinforcement. , 2012 .

[20]  G. Schoenbaum,et al.  Model‐based learning and the contribution of the orbitofrontal cortex to the model‐free world , 2012, The European journal of neuroscience.

[21]  B. Everitt,et al.  Differential Roles of the Dorsolateral and Midlateral Striatum in Punished Cocaine Seeking , 2012, The Journal of Neuroscience.

[22]  A. Graybiel,et al.  Habit learning is associated with major shifts in frequencies of oscillatory activity and synchronized spike firing in striatum , 2011, Proceedings of the National Academy of Sciences.

[23]  Ethan S. Bromberg-Martin,et al.  Dopamine in Motivational Control: Rewarding, Aversive, and Alerting , 2010, Neuron.

[24]  D. H. Root,et al.  Absence of cue-evoked firing in rat dorsolateral striatum neurons , 2010, Behavioural Brain Research.

[25]  Xin Jin,et al.  Start/stop signals emerge in nigrostriatal circuits during sequence learning , 2010, Nature.

[26]  A. Graybiel,et al.  Differential Dynamics of Activity Changes in Dorsolateral and Dorsomedial Striatal Loops during Learning , 2010, Neuron.

[27]  A. Graybiel,et al.  Neural representation of time in cortico-basal ganglia circuits , 2009, Proceedings of the National Academy of Sciences.

[28]  David J. Barker,et al.  Decreased Firing of Striatal Neurons Related to Licking during Acquisition and Overtraining of a Licking Task , 2009, The Journal of Neuroscience.

[29]  A. Graybiel,et al.  Stable encoding of task structure coexists with flexible coding of task events in sensorimotor striatum. , 2009, Journal of neurophysiology.

[30]  M. Laubach,et al.  Neuronal correlates of instrumental learning in the dorsal striatum. , 2009, Journal of neurophysiology.

[31]  B. Balleine,et al.  A specific role for posterior dorsolateral striatum in human habit learning , 2009, The European journal of neuroscience.

[32]  M. Packard Exhumed from thought: Basal ganglia and response learning in the plus-maze , 2009, Behavioural Brain Research.

[33]  B. Balleine,et al.  The integrative function of the basal ganglia in instrumental conditioning , 2009, Behavioural Brain Research.

[34]  H. Eichenbaum,et al.  Striatal versus hippocampal representations during win-stay maze performance. , 2009, Journal of neurophysiology.

[35]  B. Balleine,et al.  Reward‐guided learning beyond dopamine in the nucleus accumbens: the integrative functions of cortico‐basal ganglia networks , 2008, The European journal of neuroscience.

[36]  A. Graybiel Habits, rituals, and the evaluative brain. , 2008, Annual review of neuroscience.

[37]  P. Janak,et al.  Inactivation of the Lateral But Not Medial Dorsal Striatum Eliminates the Excitatory Impact of Pavlovian Stimuli on Instrumental Responding , 2007, The Journal of Neuroscience.

[38]  JaneR . Taylor,et al.  Bidirectional modulation of goal-directed actions by prefrontal cortical dopamine. , 2007, Cerebral cortex.

[39]  M. West,et al.  Changes in activity of the striatum during formation of a motor habit , 2007, The European journal of neuroscience.

[40]  N. Volkow,et al.  Cocaine Cues and Dopamine in Dorsal Striatum: Mechanism of Craving in Cocaine Addiction , 2006, The Journal of Neuroscience.

[41]  H. Yin,et al.  The role of the basal ganglia in habit formation , 2006, Nature Reviews Neuroscience.

[42]  A. Graybiel,et al.  Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories , 2005, Nature.

[43]  P. Tresco,et al.  Response of brain tissue to chronically implanted neural electrodes , 2005, Journal of Neuroscience Methods.

[44]  A M Graybiel,et al.  Time-varying covariance of neural activities recorded in striatum and frontal cortex as monkeys perform sequential-saccade tasks. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[45]  Wolfram Schultz,et al.  Relative reward processing in primate striatum , 2005, Experimental Brain Research.

[46]  H. Eichenbaum,et al.  Oscillatory Entrainment of Striatal Neurons in Freely Moving Rats , 2004, Neuron.

[47]  J. W. Aldridge,et al.  Basal ganglia neural mechanisms of natural movement sequences. , 2004, Canadian journal of physiology and pharmacology.

[48]  A. Graybiel,et al.  Representation of Action Sequence Boundaries by Macaque Prefrontal Cortical Neurons , 2003, Science.

[49]  S. Killcross,et al.  Coordination of actions and habits in the medial prefrontal cortex of rats. , 2003, Cerebral cortex.

[50]  C. I. Connolly,et al.  Building neural representations of habits. , 1999, Science.

[51]  J. Hollerman,et al.  Influence of reward expectation on behavior-related neuronal activity in primate striatum. , 1998, Journal of neurophysiology.

[52]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[53]  O. Hikosaka,et al.  Functional properties of monkey caudate neurons. III. Activities related to expectation of target and reward. , 1989, Journal of neurophysiology.

[54]  A. Dickinson Actions and habits: the development of behavioural autonomy , 1985 .

[55]  J. Pearce,et al.  A model for Pavlovian learning: variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980, Psychological review.

[56]  W. Schultz,et al.  Behavioral theories and the neurophysiology of reward. , 2006, Annual review of psychology.

[57]  P. Dayan,et al.  Actions , Policies , Values , and the Basal Ganglia , 2005 .

[58]  W. Schultz,et al.  Responses to reward in monkey dorsal and ventral striatum , 2004, Experimental Brain Research.

[59]  Georgios A. Keliris,et al.  Neuronal Activity in the Rodent Dorsal Striatum in Sequential Navigation : Separation of Spatial and Reward Responses on the Multiple T Task , 2004 .