Interfacing behavioral and neural circuit models for habit formation

Habits are an important mechanism by which organisms can automate the control of behavior to alleviate cognitive demand. However, transitions to habitual control are risky because they lead to inflexible responding in the face of change. The question of how the brain controls transitions into habit is thus an intriguing one. How do we regulate when our repeated actions become automated? When is it advantageous or disadvantageous to release actions from cognitive control? Decades of research have identified a variety of methods for eliciting habitual responding in animal models. Progress has also been made to understand which brain areas and neural circuits control transitions into habit. Here, I discuss existing research on behavioral and neural circuit models for habit formation (with an emphasis on striatal circuits), and discuss strategies for combining information from different paradigms and levels of analysis to prompt further progress in the field.

[1]  B. Everitt,et al.  Hierarchical recruitment of phasic dopamine signaling in the striatum during the progression of cocaine use , 2012, Proceedings of the National Academy of Sciences.

[2]  R. Costa,et al.  Orbitofrontal and striatal circuits dynamically encode the shift between goal-directed and habitual actions , 2013, Nature Communications.

[3]  T. Robbins,et al.  Enhanced Avoidance Habits in Obsessive-Compulsive Disorder , 2014, Biological Psychiatry.

[4]  Ashesh K Dhawale,et al.  The basal ganglia can control learned motor sequences independently of motor cortex , 2019 .

[5]  D. H. Root,et al.  The ventral pallidum: Subregion-specific functional anatomy and roles in motivated behaviors , 2015, Progress in Neurobiology.

[6]  Talia N. Lerner,et al.  Endocannabinoid Signaling Mediates Psychomotor Activation by Adenosine A2A Antagonists , 2010, The Journal of Neuroscience.

[7]  A. Graybiel,et al.  Stable encoding of task structure coexists with flexible coding of task events in sensorimotor striatum. , 2009, Journal of neurophysiology.

[8]  Graham C. Haug,et al.  Pattern of dopamine signaling during aversive events predicts active avoidance learning , 2019, Proceedings of the National Academy of Sciences.

[9]  Jordan B. Logue,et al.  Lesions of the Patch Compartment of Dorsolateral Striatum Disrupt Stimulus–Response Learning , 2019, Neuroscience.

[10]  Robert M Sears,et al.  The birth, death and resurrection of avoidance: a reconceptualization of a troubled paradigm , 2016, Molecular Psychiatry.

[11]  B. Knowlton,et al.  Enhanced Avoidance Habits in Relation to History of Early-Life Stress , 2019, Front. Psychol..

[12]  Liqun Luo,et al.  Topological Organization of Ventral Tegmental Area Connectivity Revealed by Viral-Genetic Dissection of Input-Output Relations , 2019, Cell reports.

[13]  JM Tepper,et al.  GABAA receptor-mediated inhibition of rat substantia nigra dopaminergic neurons by pars reticulata projection neurons , 1995, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  Mark A. Rossi,et al.  Methods for Studying Habitual Behavior in Mice , 2012, Current protocols in neuroscience.

[15]  Zhi-Li Huang,et al.  Optogenetic Activation of Adenosine A2A Receptor Signaling in the Dorsomedial Striatopallidal Neurons Suppresses Goal-Directed Behavior , 2016, Neuropsychopharmacology.

[16]  William W. Taylor,et al.  Dorsolateral Striatum Engagement Interferes with Early Discrimination Learning , 2018, Cell reports.

[17]  Edward A. Stern,et al.  Birdbrains could teach basal ganglia research a new song , 2005, Trends in Neurosciences.

[18]  Bernard W. Balleine,et al.  Actions, Action Sequences and Habits: Evidence That Goal-Directed and Habitual Action Control Are Hierarchically Organized , 2013, PLoS Comput. Biol..

[19]  Xin Jin,et al.  Basal Ganglia Subcircuits Distinctively Encode the Parsing and Concatenation of Action Sequences , 2014, Nature Neuroscience.

[20]  Jennifer A. Mangels,et al.  A Neostriatal Habit Learning System in Humans , 1996, Science.

[21]  S. Gourley,et al.  Involvement of the rodent prelimbic and medial orbitofrontal cortices in goal‐directed action: A brief review , 2019, Journal of neuroscience research.

[22]  M. Delong,et al.  Primate models of movement disorders of basal ganglia origin , 1990, Trends in Neurosciences.

[23]  B. Everitt,et al.  Involvement of the Dorsal Striatum in Cue-Controlled Cocaine Seeking , 2005, The Journal of Neuroscience.

[24]  H. Yin,et al.  Genetic Deletion of A2A Adenosine Receptors in the Striatum Selectively Impairs Habit Formation , 2009, The Journal of Neuroscience.

[25]  A. Doupe,et al.  Interruption of a basal ganglia–forebrain circuit prevents plasticity of learned vocalizations , 2000, Nature.

[26]  Shahin Rafii,et al.  Slitrk5 deficiency impairs corticostriatal circuitry and leads to obsessive-compulsive–like behaviors in mice , 2010, Nature Medicine.

[27]  Takashi Kato,et al.  Reduced striatal dopamine release during motor skill acquisition in Parkinson’s disease , 2018, PloS one.

[28]  J. Tepper,et al.  GABAergic control of substantia nigra dopaminergic neurons. , 2007, Progress in brain research.

[29]  G. Feng,et al.  Optogenetic Stimulation of Lateral Orbitofronto-Striatal Pathway Suppresses Compulsive Behaviors , 2013, Science.

[30]  B. Knowlton,et al.  Contributions of striatal subregions to place and response learning. , 2004, Learning & memory.

[31]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[32]  A. Faure,et al.  Lesion to the Nigrostriatal Dopamine System Disrupts Stimulus-Response Habit Formation , 2005, The Journal of Neuroscience.

[33]  D. M. Smith,et al.  Partial ablation of mu-opioid receptor rich striosomes produces deficits on a motor-skill learning task , 2009, Neuroscience.

[34]  Xin Jin,et al.  Start/stop signals emerge in nigrostriatal circuits during sequence learning , 2010, Nature.

[35]  L. Green,et al.  Suppression of cocaine self-administration in monkeys: effects of delayed punishment , 2012, Psychopharmacology.

[36]  B. Balleine,et al.  The role of the dorsomedial striatum in instrumental conditioning , 2005, The European journal of neuroscience.

[37]  Joshua L. Plotkin,et al.  Dopamine and synaptic plasticity in dorsal striatal circuits controlling action selection , 2009, Current Opinion in Neurobiology.

[38]  S. Ikemoto Dopamine reward circuitry: Two projection systems from the ventral midbrain to the nucleus accumbens–olfactory tubercle complex , 2007, Brain Research Reviews.

[39]  B. Balleine,et al.  Reduced goal‐directed action control in autism spectrum disorder , 2016, Autism research : official journal of the International Society for Autism Research.

[40]  T. Robinson,et al.  Are Cocaine-Seeking “Habits” Necessary for the Development of Addiction-Like Behavior in Rats? , 2017, The Journal of Neuroscience.

[41]  C. Gremel,et al.  Fractionating the all‐or‐nothing definition of goal‐directed and habitual decision‐making , 2020, Journal of neuroscience research.

[42]  J. Wickens,et al.  An open cortico-basal ganglia loop allows limbic control over motor output via the nigrothalamic pathway , 2019, eLife.

[43]  F. Woodward Hopf,et al.  Rescuing cocaine-induced prefrontal cortex hypoactivity prevents compulsive cocaine seeking , 2013, Nature.

[44]  B. Knowlton,et al.  Learning and memory functions of the Basal Ganglia. , 2002, Annual review of neuroscience.

[45]  B. Balleine,et al.  Plasticity in striatopallidal projection neurons mediates the acquisition of habitual actions , 2015, The European journal of neuroscience.

[46]  Kelly R. Tan,et al.  Synergistic Nigral Output Pathways Shape Movement. , 2019, Cell reports.

[47]  Michael S. Brainard,et al.  Learning the microstructure of successful behavior , 2011, Nature Neuroscience.

[48]  B. Everitt,et al.  Increased Impulsivity Retards the Transition to Dorsolateral Striatal Dopamine Control of Cocaine Seeking , 2014, Biological Psychiatry.

[49]  David M. Lovinger,et al.  Parallel, but Dissociable, Processing in Discrete Corticostriatal Inputs Encodes Skill Learning , 2017, Neuron.

[50]  Kevin J. Miller,et al.  Habits without Values , 2016, bioRxiv.

[51]  Sachie K. Ogawa,et al.  Dopamine neurons projecting to the posterior striatum form an anatomically distinct subclass , 2015, eLife.

[52]  P. Janak,et al.  Defining the place of habit in substance use disorders , 2017, Progress in Neuro-Psychopharmacology and Biological Psychiatry.

[53]  Nikolaus R. McFarland,et al.  Striatonigrostriatal Pathways in Primates Form an Ascending Spiral from the Shell to the Dorsolateral Striatum , 2000, The Journal of Neuroscience.

[54]  T. Robbins,et al.  Drug Addiction: Updating Actions to Habits to Compulsions Ten Years On. , 2016, Annual review of psychology.

[55]  Talia N. Lerner,et al.  Intact-Brain Analyses Reveal Distinct Information Carried by SNc Dopamine Subcircuits , 2015, Cell.

[56]  Philippe Mailly,et al.  Three-Dimensional Organization of the Recurrent Axon Collateral Network of the Substantia Nigra Pars Reticulata Neurons in the Rat , 2003, The Journal of Neuroscience.

[57]  C. Paladini,et al.  Cocaine Selectively Reorganizes Excitatory Inputs to Substantia Nigra Pars Compacta Dopamine Neurons , 2017, The Journal of Neuroscience.

[58]  Kelly R. Tan,et al.  Cocaine Disinhibits Dopamine Neurons by Potentiation of GABA Transmission in the Ventral Tegmental Area , 2013, Science.

[59]  G. Koob,et al.  Individual differences in the neuropsychopathology of addiction , 2017, Dialogues in clinical neuroscience.

[60]  S. Dudek,et al.  Cortico-striatal synaptic defects and OCD-like behaviours in Sapap3-mutant mice , 2007, Nature.

[61]  A. Graybiel,et al.  A Corticostriatal Path Targeting Striosomes Controls Decision-Making under Conflict , 2015, Cell.

[62]  A. Dickinson,et al.  Contingency Effects with Maintained Instrumental Reinforcement , 1985 .

[63]  Christopher D. Adams,et al.  The Effect of the Instrumental Training Contingency on Susceptibility to Reinforcer Devaluation , 1983 .

[64]  JaneR . Taylor,et al.  Cytoskeletal Determinants of Stimulus-Response Habits , 2013, The Journal of Neuroscience.

[65]  B. Balleine,et al.  The Role of the Nucleus Accumbens in Instrumental Conditioning: Evidence of a Functional Dissociation between Accumbens Core and Shell , 2001, The Journal of Neuroscience.

[66]  R. Costa,et al.  Frontiers in Integrative Neuroscience Integrative Neuroscience , 2022 .

[67]  B. Everitt,et al.  Cocaine Seeking Habits Depend upon Dopamine-Dependent Serial Connectivity Linking the Ventral with the Dorsal Striatum , 2008, Neuron.

[68]  K. Witt,et al.  Dissociation of Habit-Learning in Parkinson's and Cerebellar Disease , 2002, Journal of Cognitive Neuroscience.

[69]  R. Costa,et al.  Dopamine neuron activity before action initiation gates and invigorates future movements , 2018, Nature.

[70]  Ashesh K Dhawale,et al.  Motor Cortex Is Required for Learning but Not for Executing a Motor Skill , 2015, Neuron.

[71]  S. Lammel,et al.  Nucleus Accumbens Subnuclei Regulate Motivated Behavior via Direct Inhibition and Disinhibition of VTA Dopamine Subpopulations , 2018, Neuron.

[72]  Justin K. O’Hare,et al.  Pathway-Specific Striatal Substrates for Habitual Behavior , 2016, Neuron.

[73]  T. Robbins,et al.  Functional neuroimaging of avoidance habits in obsessive-compulsive disorder. , 2015, The American journal of psychiatry.

[74]  L. Vanderschuren,et al.  Punishment models of addictive behavior , 2017, Current Opinion in Behavioral Sciences.

[75]  Kyle S. Smith,et al.  Reversible online control of habitual behavior by optogenetic perturbation of medial prefrontal cortex , 2012, Proceedings of the National Academy of Sciences.

[76]  A. Dickinson Actions and habits: the development of behavioural autonomy , 1985 .

[77]  Willie F. Tobin,et al.  Rapid formation and selective stabilization of synapses for enduring motor memories , 2009, Nature.

[78]  Nagaraj R. Mahajan,et al.  Distinct recruitment of dorsomedial and dorsolateral striatum erodes with extended training , 2019, eLife.

[79]  Patrick E. Rothwell,et al.  Autism-Associated Neuroligin-3 Mutations Commonly Impair Striatal Circuits to Boost Repetitive Behaviors , 2014, Cell.

[80]  Talia N. Lerner,et al.  RGS4 Is Required for Dopaminergic Control of Striatal LTD and Susceptibility to Parkinsonian Motor Deficits , 2012, Neuron.

[81]  J. N. P. Rawlins,et al.  Effects of cytotoxic nucleus accumbens lesions on instrumental conditioning in rats , 2002, Experimental Brain Research.

[82]  Sachie K. Ogawa,et al.  Whole-Brain Mapping of Direct Inputs to Midbrain Dopamine Neurons , 2012, Neuron.

[83]  Trevor W. Robbins,et al.  High Impulsivity Predicts the Switch to Compulsive Cocaine-Taking , 2008, Science.

[84]  K. Deisseroth,et al.  Endocannabinoid Modulation of Orbitostriatal Circuits Gates Habit Formation , 2016, Neuron.

[85]  B. Balleine,et al.  Blockade of NMDA receptors in the dorsomedial striatum prevents action–outcome learning in instrumental conditioning , 2005, The European journal of neuroscience.

[86]  R. Costa,et al.  Endocannabinoid Signaling is Critical for Habit Formation , 2007, Frontiers in integrative neuroscience.

[87]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[88]  D. Lovinger,et al.  Dynamic reorganization of striatal circuits during the acquisition and consolidation of a skill , 2009, Nature Neuroscience.

[89]  P. Janak,et al.  Lever Insertion as a Salient Stimulus Promoting Insensitivity to Outcome Devaluation , 2017, Front. Integr. Neurosci..

[90]  B. Balleine,et al.  Effects of ibotenic acid lesions of the Nucleus Accumbens on instrumental action , 1994, Behavioural Brain Research.

[91]  Liqun Luo,et al.  Circuit Architecture of VTA Dopamine Neurons Revealed by Systematic Input-Output Mapping , 2015, Cell.

[92]  Bernard W Balleine,et al.  The Acquisition of Goal-Directed Actions Generates Opposing Plasticity in Direct and Indirect Pathways in Dorsomedial Striatum , 2014, The Journal of Neuroscience.

[93]  B. Everitt,et al.  Compulsive Alcohol Seeking Results from a Failure to Disengage Dorsolateral Striatal Control over Behavior , 2019, The Journal of Neuroscience.

[94]  A. Graybiel,et al.  Differential Dynamics of Activity Changes in Dorsolateral and Dorsomedial Striatal Loops during Learning , 2010, Neuron.

[95]  B. Everitt,et al.  Heroin seeking becomes dependent on dorsal striatal dopaminergic mechanisms and can be decreased by N‐acetylcysteine , 2018, The European journal of neuroscience.

[96]  B. Everitt,et al.  Drug Seeking Becomes Compulsive After Prolonged Cocaine Self-Administration , 2004, Science.

[97]  S. T. Kitai,et al.  Electrophysiological and immunocytochemical characterization of GABA and dopamine neurons in the substantia nigra of the rat , 1997, Neuroscience.

[98]  Anatol C. Kreitzer,et al.  Control of Basal Ganglia Output by Direct and Indirect Pathway Projection Neurons , 2013, The Journal of Neuroscience.

[99]  K. Deisseroth,et al.  Input-specific control of reward and aversion in the ventral tegmental area , 2012, Nature.

[100]  M. Howe,et al.  Rapid signaling in distinct dopaminergic axons during locomotion and reward , 2016, Nature.

[101]  Ashesh K Dhawale,et al.  The Role of Variability in Motor Learning. , 2017, Annual review of neuroscience.

[102]  Kyle S. Smith,et al.  Habit formation coincides with shifts in reinforcement representations in the sensorimotor striatum. , 2016, Journal of neurophysiology.

[103]  David Belin,et al.  Evidence for Addiction-like Behavior in the Rat , 2004, Science.

[104]  Barry J. Everitt,et al.  Compulsive drug seeking by rats under punishment: effects of drug taking history , 2007, Psychopharmacology.

[105]  B. Balleine,et al.  Lesions of dorsolateral striatum preserve outcome expectancy but disrupt habit formation in instrumental learning , 2004, The European journal of neuroscience.

[106]  H. Yin,et al.  The role of the basal ganglia in habit formation , 2006, Nature Reviews Neuroscience.

[107]  N. Uchida,et al.  Dopamine neurons projecting to the posterior striatum reinforce avoidance of threatening stimuli , 2018, Nature Neuroscience.

[108]  Peter Redgrave,et al.  Reduced habit-driven errors in Parkinson’s Disease , 2019, Scientific Reports.

[109]  O. Hikosaka,et al.  Two types of dopamine neuron distinctly convey positive and negative motivational signals , 2009, Nature.

[110]  K. Deisseroth,et al.  Repeated Cortico-Striatal Stimulation Generates Persistent OCD-Like Behavior , 2013, Science.

[111]  P. Greengard,et al.  Dichotomous Dopaminergic Control of Striatal Synaptic Plasticity , 2008, Science.

[112]  Samuel D. Gale,et al.  A Basal Ganglia Pathway Drives Selective Auditory Responses in Songbird Dopaminergic Neurons via Disinhibition , 2010, The Journal of Neuroscience.

[113]  J. Penney,et al.  The functional anatomy of basal ganglia disorders , 1989, Trends in Neurosciences.

[114]  A. Rivera,et al.  Selective ablation of striatal striosomes produces the deregulation of dopamine nigrostriatal pathway , 2018, PloS one.

[115]  B. Balleine,et al.  Motivational control of goal-directed action , 1994 .

[116]  Bernard W Balleine,et al.  The Ventral Striato-Pallidal Pathway Mediates the Effect of Predictive Learning on Choice between Goal-Directed Actions , 2013, The Journal of Neuroscience.

[117]  J. Goldberg,et al.  Dopamine neurons encode performance error in singing birds , 2016, Science.

[118]  E. Callaway,et al.  Genetic-Based Dissection Unveils the Inputs and Outputs of Striatal Patch and Matrix Compartments , 2016, Neuron.

[119]  Talia N. Lerner,et al.  Neuromodulatory control of striatal plasticity and behavior , 2011, Current Opinion in Neurobiology.

[120]  G. Feng,et al.  Shank3 mutant mice display autistic-like behaviours and striatal dysfunction , 2011, Nature.

[121]  M. Fan,et al.  Compulsive sucrose- and cocaine-seeking behaviors in male and female Wistar rats , 2018, Psychopharmacology.

[122]  M. Sur,et al.  Two-photon imaging in mice shows striosomes and matrix have overlapping but differential reinforcement-related responses , 2017, eLife.

[123]  Shinsuke Shimojo,et al.  Neural Computations Underlying Arbitration between Model-Based and Model-free Learning , 2013, Neuron.

[124]  Stephen A. Allsop,et al.  Decoding Neural Circuits that Control Compulsive Sucrose Seeking , 2015, Cell.

[125]  A. Graybiel,et al.  Basal Ganglia Disorders Associated with Imbalances in the Striatal Striosome and Matrix Compartments , 2011, Front. Neuroanat..