Adolescents exhibit reduced Pavlovian biases on instrumental learning

Multiple learning systems allow individuals to flexibly respond to opportunities and challenges present in the environment. An evolutionarily conserved “Pavlovian” learning mechanism couples valence and action, promoting a tendency to approach cues associated with reward and to inhibit action in the face of anticipated punishment. Although this default response system may be adaptive, these hard-wired reactions can hinder the ability to learn flexible “instrumental” actions in pursuit of a goal. Such constraints on behavioral flexibility have been studied extensively in adults. However, the extent to which these valence-specific response tendencies bias instrumental learning across development remains poorly characterized. Here, we show that while Pavlovian response biases constrain flexible action learning in children and adults, these biases are attenuated in adolescents. This adolescent-specific reduction in Pavlovian bias may promote unbiased exploration of approach and avoidance responses, facilitating the discovery of rewarding behavior in the many novel contexts that adolescents encounter.

[1]  W. Estes Discriminative conditioning. I. A discriminative property of conditioned anticipation. , 1943 .

[2]  M. Schoog [Effects of D-cycloserine]. , 1958, Arzneimittel-Forschung.

[3]  K. Breland,et al.  The misbehavior of organisms. , 1961 .

[4]  R. Rescorla,et al.  Two-process learning theory: Relationships between Pavlovian conditioning and instrumental learning. , 1967, Psychological review.

[5]  D. R. Williams,et al.  Auto-maintenance in the pigeon: sustained pecking despite contingent non-reinforcement. , 1969, Journal of the experimental analysis of behavior.

[6]  R. Bolles Species-specific defense reactions and avoidance learning. , 1970 .

[7]  H. Akaike A new look at the statistical model identification , 1974 .

[8]  R. Sprott,et al.  Effects of age and genotype on acquisition of an active avoidance response in mice. , 1975, Developmental psychobiology.

[9]  R. Bauer Ontogeny of two-way avoidance in male and female rats. , 1978, Developmental psychobiology.

[10]  P. Holland,et al.  Differential effects of omission contingencies on various components of Pavlovian appetitive conditioned responding in rats. , 1979, Journal of experimental psychology. Animal behavior processes.

[11]  C. Rovee-Collier,et al.  The economics of infancy: a review of conjugate reinforcement. , 1979, Advances in child development and behavior.

[12]  J. Gray,et al.  Précis of The neuropsychology of anxiety: An enquiry into the functions of the septo-hippocampal system , 1982, Behavioral and Brain Sciences.

[13]  P. Lovibond Facilitation of instrumental behavior by a Pavlovian appetitive conditioned stimulus. , 1983 .

[14]  Richard S. Sutton,et al.  Neuronlike adaptive elements that can solve difficult learning control problems , 1983, IEEE Transactions on Systems, Man, and Cybernetics.

[15]  A. Dickinson Actions and habits: the development of behavioural autonomy , 1985 .

[16]  W. Hershberger An approach through the looking-glass , 1986 .

[17]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[18]  B. Balleine,et al.  Motivational control of goal-directed action , 1994 .

[19]  A. Barto,et al.  Adaptive Critics and the Basal Ganglia , 1994 .

[20]  Martin H. Teicher,et al.  Evidence for dopamine receptor pruning between adolescence and adulthood in striatum but not nucleus accumbens. , 1995, Brain research. Developmental brain research.

[21]  Martin H. Teicher,et al.  Sex differences in dopamine receptor overproduction and elimination , 1997, Neuroreport.

[22]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[23]  A. Toga,et al.  In vivo evidence for post-adolescent brain maturation in frontal and striatal regions , 1999, Nature Neuroscience.

[24]  R. Mailman,et al.  Developmental Regulation of the Dopamine D1 Receptor in Human Caudate and Putamen , 1999, Neuropsychopharmacology.

[25]  S. Takashima,et al.  Developmental and age-related changes of dopamine transporter, and dopamine D1 and D2 receptors in human basal ganglia , 1999, Brain Research.

[26]  B. Carter,et al.  Meta-analysis of cue-reactivity in addiction research. , 1999, Addiction.

[27]  L. Spear The adolescent brain and age-related behavioral manifestations , 2000, Neuroscience & Biobehavioral Reviews.

[28]  Martin H. Teicher,et al.  Dopamine receptor pruning in prefrontal cortex during the periadolescent period in rats , 2000, Synapse.

[29]  A. Dickinson,et al.  The neuropsychological basis of addictive behaviour , 2001, Brain Research Reviews.

[30]  J. March,et al.  Adaptation as Information Restriction: The Hot Stove Effect , 2001 .

[31]  P. Dayan,et al.  Reward, Motivation, and Reinforcement Learning , 2002, Neuron.

[32]  B. Everitt,et al.  Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex , 2002, Neuroscience & Biobehavioral Reviews.

[33]  P. Holland Relations between Pavlovian-instrumental transfer and reinforcer devaluation. , 2004, Journal of experimental psychology. Animal behavior processes.

[34]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[35]  R. Dahl Adolescent Brain Development: A Period of Vulnerabilities and Opportunities. Keynote Address , 2004, Annals of the New York Academy of Sciences.

[36]  Thomas F. Nugent,et al.  Dynamic mapping of human cortical development during childhood through early adulthood. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[37]  Olga V. Demler,et al.  Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. , 2005, Archives of general psychiatry.

[38]  H. Yin,et al.  The role of the basal ganglia in habit formation , 2006, Nature Reviews Neuroscience.

[39]  C. Liston,et al.  Frontostriatal microstructure modulates efficient recruitment of cognitive control. , 2006, Cerebral cortex.

[40]  A. Diamond The Early Development of Executive Functions. , 2006 .

[41]  B. McEwen,et al.  Stress and the Adolescent Brain , 2006, Annals of the New York Academy of Sciences.

[42]  S. Bunge,et al.  Neurodevelopmental changes in working memory and cognitive control , 2007, Current Opinion in Neurobiology.

[43]  J. Russell,et al.  The control of instrumental action following outcome devaluation in young children aged between 1 and 4 years. , 2008, Journal of experimental psychology. General.

[44]  B. Balleine,et al.  The Neural Mechanisms Underlying the Influence of Pavlovian Cues on Human Decision Making , 2008, The Journal of Neuroscience.

[45]  L. Steinberg A Social Neuroscience Perspective on Adolescent Risk-Taking. , 2008, Developmental review : DR.

[46]  T. Paus,et al.  Why do many psychiatric disorders emerge during adolescence? , 2008, Nature Reviews Neuroscience.

[47]  S. Mineka,et al.  The relevance of recent developments in classical conditioning to understanding the etiology and maintenance of anxiety disorders. , 2008, Acta psychologica.

[48]  P. Dayan,et al.  Human Pavlovian–Instrumental Transfer , 2008, The Journal of Neuroscience.

[49]  G. Gredebäck,et al.  Goal directedness and decision making in infants. , 2009, Developmental psychology.

[50]  B. McEwen,et al.  Effects of stress throughout the lifespan on the brain, behaviour and cognition , 2009, Nature Reviews Neuroscience.

[51]  B. Luna Developmental changes in cognitive control through adolescence. , 2009, Advances in child development and behavior.

[52]  P. Dayan,et al.  A Bayesian formulation of behavioral control , 2009, Cognition.

[53]  T. Robbins,et al.  Reconciling the Role of Serotonin in Behavioral Inhibition and Aversion: Acute Tryptophan Depletion Abolishes Punishment-Induced Inhibition in Humans , 2009, The Journal of Neuroscience.

[54]  T. Maia Two-factor theory, the actor-critic model, and conditioned avoidance , 2010, Learning & behavior.

[55]  J. Kim,et al.  Impaired Extinction Retention in Adolescent Rats: Effects of D-Cycloserine , 2010, Neuropsychopharmacology.

[56]  B. Casey,et al.  Adolescence: What Do Transmission, Transition, and Translation Have to Do with It? , 2010, Neuron.

[57]  L. Somerville,et al.  Developmental neurobiology of cognitive control and motivational systems , 2010, Current Opinion in Neurobiology.

[58]  B. Balleine,et al.  Human and Rodent Homologies in Action Control: Corticostriatal Determinants of Goal-Directed and Habitual Action , 2010, Neuropsychopharmacology.

[59]  Joseph E LeDoux,et al.  The role of amygdala nuclei in the expression of auditory signaled two-way active avoidance in rats. , 2010, Learning & memory.

[60]  T. White,et al.  Neurobehavioral evidence for changes in dopamine system activity during adolescence , 2010, Neuroscience & Biobehavioral Reviews.

[61]  P. Dayan,et al.  Behavioral/systems/cognitive Action Dominates Valence in Anticipatory Representations in the Human Striatum and Dopaminergic Midbrain , 2010 .

[62]  P. Dayan,et al.  Model-based influences on humans’ choices and striatal prediction errors , 2011, Neuron.

[63]  L. Spear,et al.  Autoshaping in adolescence enhances sign-tracking behavior in adulthood: Impact on ethanol consumption , 2011, Pharmacology Biochemistry and Behavior.

[64]  L. Spear,et al.  Amphetamine-induced incentive sensitization of sign-tracking behavior in adolescent and adult female rats. , 2011, Behavioral neuroscience.

[65]  P. Dayan,et al.  Opponency Revisited: Competition and Cooperation Between Dopamine and Serotonin , 2010, Neuropsychopharmacology.

[66]  Peter Dayan,et al.  Bonsai Trees in Your Head: How the Pavlovian System Sculpts Goal-Directed Choices by Pruning Decision Trees , 2012, PLoS Comput. Biol..

[67]  P. Dayan,et al.  Action controls dopaminergic enhancement of reward representations , 2012, Proceedings of the National Academy of Sciences.

[68]  Michael X. Cohen,et al.  Striatum-medial prefrontal cortex connectivity predicts developmental changes in reinforcement learning. , 2012, Cerebral cortex.

[69]  Raymond J. Dolan,et al.  Go and no-go learning in reward and punishment: Interactions between affect and effect , 2012, NeuroImage.

[70]  R. Dolan,et al.  Dopamine Enhances Model-Based over Model-Free Choice Behavior , 2012, Neuron.

[71]  Catherine A. Hartley,et al.  Altered fear learning across development in both mouse and human , 2012, Proceedings of the National Academy of Sciences.

[72]  P. Glimcher,et al.  Adolescents’ risk-taking behavior is driven by tolerance to ambiguity , 2012, Proceedings of the National Academy of Sciences.

[73]  G. Glover,et al.  The Medial Prefrontal Cortex and the Emergence of Self-Conscious Emotion in Adolescence , 2013, Psychological science.

[74]  P. Dayan,et al.  Differential, but not opponent, effects of l-DOPA and citalopram on action learning with reward and punishment , 2013, Psychopharmacology.

[75]  L. Spear,et al.  Environmental manipulations alter age differences in attribution of incentive salience to reward-paired cues , 2013, Behavioural Brain Research.

[76]  Ian W. Eisenberg,et al.  Frontal Theta Overrides Pavlovian Learning Biases , 2013, The Journal of Neuroscience.

[77]  R. Berman,et al.  Longitudinal four-dimensional mapping of subcortical anatomy in human development , 2014, Proceedings of the National Academy of Sciences.

[78]  N. Fox,et al.  Fear conditioning and extinction across development: Evidence from human studies and animal models , 2014, Biological Psychology.

[79]  Joseph E LeDoux,et al.  Heterogeneity in signaled active avoidance learning: substantive and methodological relevance of diversity in instrumental defensive responses to threat cues , 2014, Front. Syst. Neurosci..

[80]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[81]  S. Blakemore,et al.  Adolescence as a Sensitive Period of Brain Development , 2015, Trends in Cognitive Sciences.

[82]  Peter Dayan,et al.  Interplay of approximate planning strategies , 2015, Proceedings of the National Academy of Sciences.

[83]  D. Bates,et al.  Linear Mixed-Effects Models using 'Eigen' and S4 , 2015 .

[84]  Catherine A. Hartley,et al.  Sensitive Periods in Affective Development: Nonlinear Maturation of Fear Learning , 2015, Neuropsychopharmacology.

[85]  Norbert Kathmann,et al.  Don't Think, Just Feel the Music: Individuals with Strong Pavlovian-to-Instrumental Transfer Effects Rely Less on Model-based Reinforcement Learning , 2016, Journal of Cognitive Neuroscience.

[86]  Eveline A. Crone,et al.  Structural brain development between childhood and adulthood: Convergence across four longitudinal samples , 2016, NeuroImage.

[87]  Eveline A. Crone,et al.  Testing a dual-systems model of adolescent brain development using resting-state connectivity analyses , 2016, NeuroImage.

[88]  J. O'Doherty,et al.  Multiple Systems for the Motivational Control of Behavior and Associated Neural Substrates in Humans. , 2015, Current topics in behavioral neurosciences.

[89]  Catherine A. Hartley,et al.  From Creatures of Habit to Goal-Directed Learners , 2016, Psychological science.

[90]  R. Dolan,et al.  Prospective and Pavlovian mechanisms in aversive behaviour , 2016, Cognition.

[91]  P. Dayan,et al.  Safety out of control: dopamine and defence , 2016, Behavioral and Brain Functions.

[92]  H. Walter,et al.  Pavlovian‐to‐instrumental transfer effects in the nucleus accumbens relate to relapse in alcohol dependence , 2016, Addiction biology.

[93]  M. Frank,et al.  University of Birmingham Catecholaminergic challenge uncovers distinct Pavlovian and instrumental mechanisms of motivated (in)action , 2017 .

[94]  Peter Stone,et al.  Reinforcement learning , 2019, Scholarpedia.

[95]  P. Dayan,et al.  Algorithms for survival: a comparative perspective on emotions , 2017, Nature Reviews Neuroscience.

[96]  P. Dayan,et al.  Modeling Avoidance in Mood and Anxiety Disorders Using Reinforcement Learning , 2017, Biological Psychiatry.

[97]  C. Hartley,et al.  Agency and the Calibration of Motivated Behavior , 2017, Trends in Cognitive Sciences.

[98]  D. Bucci,et al.  Increased sign‐tracking behavior in adolescent rats , 2017, Developmental psychobiology.

[99]  P. Dayan,et al.  The Neural Basis of Aversive Pavlovian Guidance during Planning , 2017, The Journal of Neuroscience.

[100]  Katherine E. Powers,et al.  Development of self-protective biases in response to social evaluative feedback , 2017, Proceedings of the National Academy of Sciences.

[101]  R. Dolan,et al.  The impact of traumatic stress on Pavlovian biases , 2017, Psychological Medicine.

[102]  Khrista Boylan,et al.  Mechanisms of Change. , 2017, Journal of the Canadian Academy of Child and Adolescent Psychiatry = Journal de l'Academie canadienne de psychiatrie de l'enfant et de l'adolescent.

[103]  Catherine A. Hartley,et al.  Cognitive components underpinning the development of model-based learning , 2016, Developmental Cognitive Neuroscience.

[104]  Daniel S. Hoops,et al.  Making Dopamine Connections in Adolescence , 2017, Trends in Neurosciences.

[105]  Alexander S. Rich,et al.  The Limits of Learning: Exploration, Generalization, and the Development of Learning Traps , 2018, Journal of experimental psychology. General.

[106]  Joseph E LeDoux,et al.  Surviving threats: neural circuit and computational implications of a new taxonomy of defensive behaviour , 2018, Nature Reviews Neuroscience.

[107]  R. Dolan,et al.  Dorsal striatal dopamine D1 receptor availability predicts an instrumental bias in action learning , 2018, Proceedings of the National Academy of Sciences.

[108]  Jessica I. Määttä,et al.  Frontal network dynamics reflect neurocomputational mechanisms for reducing maladaptive biases in motivated action , 2018, PLoS biology.

[109]  B. Kolb,et al.  DCC Receptors Drive Prefrontal Cortex Maturation by Determining Dopamine Axon Targeting in Adolescence , 2018, Biological Psychiatry.

[110]  Daniel S. Hoops,et al.  Disorders of the Nervous System Dopamine Development in the Mouse Orbital Prefrontal Cortex Is Protracted and Sensitive to Amphetamine in Adolescence , 2018 .

[111]  Catherine A. Hartley,et al.  The development of goal-directed decision-making , 2018 .

[112]  Michael Moutoussis,et al.  Change, stability, and instability in the Pavlovian guidance of behaviour from adolescence to young adulthood , 2018, PLoS Comput. Biol..

[113]  Erik C. Nook,et al.  The Nonlinear Development of Emotion Differentiation: Granular Emotional Experience Is Low in Adolescence , 2018, Psychological science.

[114]  T. Verstynen,et al.  Developmental Changes in the Integration of Affective and Cognitive Corticostriatal Pathways are Associated with Reward-Driven Behavior , 2018, Cerebral cortex.

[115]  Gail M. Rosenbaum,et al.  Developmental perspectives on risky and impulsive choice , 2018, Philosophical Transactions of the Royal Society B.

[116]  Heidi C. Meyer,et al.  Translating Developmental Neuroscience to Understand Risk for Psychiatric Disorders. , 2019, The American journal of psychiatry.

[117]  Hayley M. Dorfman,et al.  Controllability governs the balance between Pavlovian and instrumental action selection , 2019, Nature Communications.

[118]  P. Hoffman Reductions in prefrontal activation predict off-topic utterances during speech production , 2018, bioRxiv.