A Dual Operator View of Habitual Behavior Reflecting Cortical and Striatal Dynamics

Habits are notoriously difficult to break and, if broken, are usually replaced by new routines. To examine the neural basis of these characteristics, we recorded spike activity in cortical and striatal habit sites as rats learned maze tasks. Overtraining induced a shift from purposeful to habitual behavior. This shift coincided with the activation of neuronal ensembles in the infralimbic neocortex and the sensorimotor striatum, which became engaged simultaneously but developed changes in spike activity with distinct time courses and stability. The striatum rapidly acquired an action-bracketing activity pattern insensitive to reward devaluation but sensitive to running automaticity. A similar pattern developed in the upper layers of the infralimbic cortex, but it formed only late during overtraining and closely tracked habit states. Selective optogenetic disruption of infralimbic activity during overtraining prevented habit formation. We suggest that learning-related spiking dynamics of both striatum and neocortex are necessary, as dual operators, for habit crystallization.

[1]  Catherine Stamoulis,et al.  Advance cueing produces enhanced action-boundary patterns of spike activity in the sensorimotor striatum. , 2011, Journal of neurophysiology.

[2]  A. Graybiel,et al.  Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories , 2005, Nature.

[3]  S. Hyman,et al.  Neural mechanisms of addiction: the role of reward-related learning and memory. , 2006, Annual review of neuroscience.

[4]  Dai Watanabe,et al.  Neural Coding of Syntactic Structure in Learned Vocalizations in the Songbird , 2011, The Journal of Neuroscience.

[5]  Paul E Holtzheimer,et al.  Deep brain stimulation for psychiatric disorders. , 2011, Annual review of neuroscience.

[6]  B. Balleine,et al.  Amygdala Central Nucleus Interacts with Dorsolateral Striatum to Regulate the Acquisition of Habits , 2012, The Journal of Neuroscience.

[7]  S. Killcross,et al.  Inactivation of the infralimbic prefrontal cortex reinstates goal-directed responding in overtrained rats , 2003, Behavioural Brain Research.

[8]  R. Douglas,et al.  Neuronal circuits of the neocortex. , 2004, Annual review of neuroscience.

[9]  H. Yin,et al.  The role of the basal ganglia in habit formation , 2006, Nature Reviews Neuroscience.

[10]  Roshan Cools,et al.  Habitual versus Goal-directed Action Control in Parkinson Disease , 2011, Journal of Cognitive Neuroscience.

[11]  Xin Jin,et al.  Start/stop signals emerge in nigrostriatal circuits during sequence learning , 2010, Nature.

[12]  N. Volkow,et al.  The neural basis of addiction: a pathology of motivation and choice. , 2005, The American journal of psychiatry.

[13]  J. W. Aldridge,et al.  Basal ganglia neural mechanisms of natural movement sequences. , 2004, Canadian journal of physiology and pharmacology.

[14]  A. Redish,et al.  Neuronal activity in the rodent dorsal striatum in sequential navigation: separation of spatial and reward responses on the multiple T task. , 2004, Journal of neurophysiology.

[15]  E. Marder Variability, compensation, and modulation in neurons and circuits , 2011, Proceedings of the National Academy of Sciences.

[16]  Michael S. Brainard,et al.  What songbirds teach us about learning , 2002, Nature.

[17]  P. Dayan,et al.  Mapping value based planning and extensively trained choice in the human brain , 2012, Nature Neuroscience.

[18]  J. Peters,et al.  Extinction circuits for fear and addiction overlap in prefrontal cortex. , 2009, Learning & memory.

[19]  M. Packard Exhumed from thought: Basal ganglia and response learning in the plus-maze , 2009, Behavioural Brain Research.

[20]  JaneR . Taylor,et al.  Bidirectional modulation of goal-directed actions by prefrontal cortical dopamine. , 2007, Cerebral cortex.

[21]  A. Graybiel,et al.  Stable encoding of task structure coexists with flexible coding of task events in sensorimotor striatum. , 2009, Journal of neurophysiology.

[22]  O. Hikosaka,et al.  Perceptual Learning, Motor Learning and Automaticity Switching from Automatic to Controlled Behavior: Cortico-basal Ganglia Mechanisms , 2022 .

[23]  Christopher D. Adams Variations in the Sensitivity of Instrumental Responding to Reinforcer Devaluation , 1982 .

[24]  T. Robbins,et al.  Neural systems of reinforcement for drug addiction: from actions to habits to compulsion , 2005, Nature Neuroscience.

[25]  H. Eichenbaum,et al.  Striatal versus hippocampal representations during win-stay maze performance. , 2009, Journal of neurophysiology.

[26]  Erin L. Rich,et al.  Rat Prefrontal Cortical Neurons Selectively Code Strategy Switches , 2009, The Journal of Neuroscience.

[27]  K. F. Muenzinger Vicarious Trial and Error at a Point of Choice: I. A General Survey of its Relation to Learning Efficiency , 1938 .

[28]  Adam Johnson,et al.  Computing motivation: Incentive salience boosts of drug or appetite states , 2008, Behavioral and Brain Sciences.

[29]  Burton S. Rosner,et al.  Neuropharmacology , 1958, Nature.

[30]  Ali Ghazizadeh,et al.  Prefrontal Cortex Mediates Extinction of Responding by Two Distinct Neural Mechanisms in Accumbens Shell , 2012, The Journal of Neuroscience.

[31]  M. West,et al.  Changes in activity of the striatum during formation of a motor habit , 2007, The European journal of neuroscience.

[32]  A. Mcgeorge,et al.  The organization of the projection from the cerebral cortex to the striatum in the rat , 1989, Neuroscience.

[33]  Michael S. Brainard,et al.  Covert skill learning in a cortical-basal ganglia circuit , 2012, Nature.

[34]  A. Graybiel Habits, rituals, and the evaluative brain. , 2008, Annual review of neuroscience.

[35]  Kyle S. Smith,et al.  Reversible online control of habitual behavior by optogenetic perturbation of medial prefrontal cortex , 2012, Proceedings of the National Academy of Sciences.

[36]  B. Balleine,et al.  The integrative function of the basal ganglia in instrumental conditioning , 2009, Behavioural Brain Research.

[37]  C. I. Connolly,et al.  Building neural representations of habits. , 1999, Science.

[38]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 2005, IEEE Transactions on Neural Networks.

[39]  Taro Kiritani,et al.  Sublayer-specific microcircuits of corticospinal and corticostriatal neurons in motor cortex , 2010, Nature Neuroscience.

[40]  Simon Killcross,et al.  Lesions of rat infralimbic cortex enhance recovery and reinstatement of an appetitive Pavlovian response. , 2004, Learning & memory.

[41]  S. Glickman,et al.  A biological theory of reinforcement. , 1967, Psychological review.

[42]  Jennifer S. Beer,et al.  Prefrontal involvement in the regulation of emotion: convergence of rat and human studies , 2006, Current Opinion in Neurobiology.

[43]  M. Laubach,et al.  Neuronal correlates of instrumental learning in the dorsal striatum. , 2009, Journal of neurophysiology.

[44]  R. O’Reilly,et al.  Separate neural substrates for skill learning and performance in the ventral and dorsal striatum , 2007, Nature Neuroscience.

[45]  P. Holland,et al.  Differential effects of two ways of devaluing the unconditioned stimulus after Pavlovian appetitive conditioning. , 1979, Journal of experimental psychology. Animal behavior processes.

[46]  S. Killcross,et al.  Coordination of actions and habits in the medial prefrontal cortex of rats. , 2003, Cerebral cortex.

[47]  B. Balleine,et al.  A specific role for posterior dorsolateral striatum in human habit learning , 2009, The European journal of neuroscience.

[48]  A. Graybiel The Basal Ganglia and Chunking of Action Repertoires , 1998, Neurobiology of Learning and Memory.

[49]  A. Graybiel,et al.  Differential Dynamics of Activity Changes in Dorsolateral and Dorsomedial Striatal Loops during Learning , 2010, Neuron.

[50]  C. Saper,et al.  Efferent projections of the infralimbic cortex of the rat , 1991, The Journal of comparative neurology.

[51]  Kyle S. Smith,et al.  A dual operator view of habitual behavior reflecting cortical and striatal dynamics. , 2013, Neuron.

[52]  E. Tolman Cognitive maps in rats and men. , 1948, Psychological review.

[53]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[54]  D. H. Root,et al.  Absence of cue-evoked firing in rat dorsolateral striatum neurons , 2010, Behavioural Brain Research.

[55]  Jonathan D. Cohen,et al.  An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. , 2005, Annual review of neuroscience.

[56]  A. Dickinson Actions and habits: the development of behavioural autonomy , 1985 .

[57]  A. Graybiel,et al.  Representation of Action Sequence Boundaries by Macaque Prefrontal Cortical Neurons , 2003, Science.

[58]  H. Eichenbaum,et al.  Oscillatory Entrainment of Striatal Neurons in Freely Moving Rats , 2004, Neuron.

[59]  P. Dayan,et al.  Actions , Policies , Values , and the Basal Ganglia , 2005 .

[60]  Jadin C. Jackson,et al.  Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. , 2007, Psychological review.

[61]  J. Csicsvari,et al.  Intracellular features predicted by extracellular recordings in the hippocampus in vivo. , 2000, Journal of neurophysiology.

[62]  F. Ervin,et al.  Appetites, aversions, and addictions: a model for visceral memory. , 1968, Recent advances in biological psychiatry.

[63]  M. West,et al.  Loss of Lever Press-Related Firing of Rat Striatal Forelimb Neurons after Repeated Sessions in a Lever Pressing Task , 1997, The Journal of Neuroscience.