Cognitive flexibility in adolescence: Neural and behavioral mechanisms of reward prediction error processing in adaptive decision making during development

Adolescence is associated with quickly changing environmental demands which require excellent adaptive skills and high cognitive flexibility. Feedback-guided adaptive learning and cognitive flexibility are driven by reward prediction error (RPE) signals, which indicate the accuracy of expectations and can be estimated using computational models. Despite the importance of cognitive flexibility during adolescence, only little is known about how RPE processing in cognitive flexibility deviates between adolescence and adulthood. In this study, we investigated the developmental aspects of cognitive flexibility by means of computational models and functional magnetic resonance imaging (fMRI). We compared the neural and behavioral correlates of cognitive flexibility in healthy adolescents (12–16 years) to adults performing a probabilistic reversal learning task. Using a modified risk-sensitive reinforcement learning model, we found that adolescents learned faster from negative RPEs than adults. The fMRI analysis revealed that within the RPE network, the adolescents had a significantly altered RPE-response in the anterior insula. This effect seemed to be mainly driven by increased responses to negative prediction errors. In summary, our findings indicate that decision making in adolescence goes beyond merely increased reward-seeking behavior and provides a developmental perspective to the behavioral and neural mechanisms underlying cognitive flexibility in the context of reinforcement learning.

[1]  H. Critchley Neural mechanisms of autonomic, affective, and cognitive integration , 2005, The Journal of comparative neurology.

[2]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[3]  W. A. Scott Cognitive Complexity and Cognitive Flexibility , 1962 .

[4]  C. Mathys,et al.  Hierarchical Prediction Errors in Midbrain and Basal Forebrain during Sensory Learning , 2013, Neuron.

[5]  Todd A. Hare,et al.  Frontostriatal Maturation Predicts Cognitive Control Failure to Appetitive Cues in Adolescents , 2011, Journal of Cognitive Neuroscience.

[6]  Serge A R B Rombouts,et al.  What motivates the adolescent? Brain regions mediating reward sensitivity across adolescence. , 2010, Cerebral cortex.

[7]  Olga V. Demler,et al.  Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. , 2005, Archives of general psychiatry.

[8]  P. Dayan,et al.  Neural Prediction Errors Reveal a Risk-Sensitive Reinforcement-Learning Process in the Human Brain , 2012, The Journal of Neuroscience.

[9]  E. Crone,et al.  Understanding adolescence as a period of social–affective engagement and goal flexibility , 2012, Nature Reviews Neuroscience.

[10]  P. Glimcher,et al.  Testing the Reward Prediction Error Hypothesis with an Axiomatic Model , 2010, The Journal of Neuroscience.

[11]  V. Menon,et al.  Saliency, switching, attention and control: a network model of insula function , 2010, Brain Structure and Function.

[12]  S. Quartz,et al.  Human Insula Activation Reflects Risk Prediction Errors As Well As Risk , 2008, The Journal of Neuroscience.

[13]  Norito Kawakami,et al.  Lifetime prevalence and age-of-onset distributions of mental disorders in the World Health Organization's World Mental Health Survey Initiative. , 2007, World psychiatry : official journal of the World Psychiatric Association.

[14]  Daniel S Pine,et al.  Cognitive inflexibility and frontal-cortical activation in pediatric obsessive-compulsive disorder. , 2010, Journal of the American Academy of Child and Adolescent Psychiatry.

[15]  Tobias U. Hauser,et al.  The feedback-related negativity (FRN) revisited: New insights into the localization, meaning and network organization , 2014, NeuroImage.

[16]  E. Weber,et al.  Affective and deliberative processes in risky choice: age differences in risk taking in the Columbia Card Task. , 2009, Journal of experimental psychology. Learning, memory, and cognition.

[17]  Steven E. Petersen,et al.  Dissociating Early and Late Error Signals in Perceptual Recognition , 2008, Journal of Cognitive Neuroscience.

[18]  Peter N. C. Mohr,et al.  Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions , 2009, Proceedings of the National Academy of Sciences.

[19]  A. Craig,et al.  How do you feel — now? The anterior insula and human awareness , 2009, Nature Reviews Neuroscience.

[20]  S. Blakemore,et al.  The role of puberty in the developing adolescent brain , 2010, Human brain mapping.

[21]  Ashley R. Smith,et al.  The Role of the Anterior Insula in Adolescent Decision Making , 2014, Developmental Neuroscience.

[22]  Antoine Bechara,et al.  The hidden island of addiction: the insula , 2009, Trends in Neurosciences.

[23]  L. Spear The adolescent brain and age-related behavioral manifestations , 2000, Neuroscience & Biobehavioral Reviews.

[24]  Michael J. Brammer,et al.  Developmental effects of reward on sustained attention networks , 2011, NeuroImage.

[25]  Bruce F. Pennington,et al.  A normative‐developmental study of executive function: A window on prefrontal function in children , 1991 .

[26]  Kristina M. Visscher,et al.  A Core System for the Implementation of Task Sets , 2006, Neuron.

[27]  T. Robbins,et al.  Cognitive Inflexibility After Prefrontal Serotonin Depletion , 2004, Science.

[28]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[29]  L. Steinberg,et al.  Peers increase adolescent risk taking by enhancing activity in the brain's reward circuitry. , 2011, Developmental science.

[30]  J. O'Doherty,et al.  The Role of the Ventromedial Prefrontal Cortex in Abstract State-Based Inference during Decision Making in Humans , 2006, The Journal of Neuroscience.

[31]  Karl J. Friston The free-energy principle: a unified brain theory? , 2010, Nature Reviews Neuroscience.

[32]  N Weiskopf,et al.  Cardiac artefact correction for human brainstem fMRI at 7T , 2009, NeuroImage.

[33]  Michael J. Brammer,et al.  Maturation of limbic corticostriatal activation and connectivity associated with developmental changes in temporal discounting , 2011, NeuroImage.

[34]  D. Denys,et al.  Dopaminergic control of cognitive flexibility in humans and animals , 2013, Front. Neurosci..

[35]  Peter Dayan,et al.  Temporal difference models describe higher-order learning in humans , 2004, Nature.

[36]  Adriana Galvan,et al.  Adolescent development of the reward system , 2022 .

[37]  Michael X. Cohen,et al.  Striatum-medial prefrontal cortex connectivity predicts developmental changes in reinforcement learning. , 2012, Cerebral cortex.

[38]  Ethan S. Bromberg-Martin,et al.  Dopamine in Motivational Control: Rewarding, Aversive, and Alerting , 2010, Neuron.

[39]  B. Berger,et al.  Catecholamine innervation of the human cerebral cortex as revealed by comparative immunohistochemistry of tyrosine hydroxylase and dopamine‐beta‐hydroxylase , 1989, The Journal of comparative neurology.

[40]  P. Tobler,et al.  Reward skewness coding in the insula independent of probability and loss. , 2011, Journal of neurophysiology.

[41]  P. Glimcher,et al.  Adolescents’ risk-taking behavior is driven by tolerance to ambiguity , 2012, Proceedings of the National Academy of Sciences.

[42]  Natalie L. Denburg,et al.  The Role of the Ventromedial Prefrontal Cortex in Purchase Intent Among Older Adults , 2016, Front. Aging Neurosci..

[43]  J. O'Doherty,et al.  Empathy for Pain Involves the Affective but not Sensory Components of Pain , 2004, Science.

[44]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[45]  Ellen Leibenluft,et al.  Neural correlates of reversal learning in severe mood dysregulation and pediatric bipolar disorder. , 2011, Journal of the American Academy of Child and Adolescent Psychiatry.

[46]  L. Steinberg A Social Neuroscience Perspective on Adolescent Risk-Taking. , 2008, Developmental review : DR.

[47]  Justin L. Vincent,et al.  Distinct brain networks for adaptive and stable task control in humans , 2007, Proceedings of the National Academy of Sciences.

[48]  K. R. Ridderinkhof,et al.  Switching between spatial stimulus-response mappings: a developmental study of cognitive flexibility. , 2004, Developmental science.

[49]  Mark W. Woolrich,et al.  Trial-Type Dependent Frames of Reference for Value Comparison , 2013, PLoS Comput. Biol..

[50]  Amir Homayoun Javadi,et al.  Adolescents Adapt More Slowly than Adults to Varying Reward Contingencies , 2014, Journal of Cognitive Neuroscience.

[51]  M. Smith,et al.  The Core System , 2017 .

[52]  M. Feenstra,et al.  Serial reversal learning and acute tryptophan depletion , 2008, Behavioural Brain Research.

[53]  G H Glover,et al.  Image‐based method for retrospective correction of physiological motion effects in fMRI: RETROICOR , 2000, Magnetic resonance in medicine.

[54]  J. Gläscher,et al.  Determining a role for ventromedial prefrontal cortex in encoding action-based value signals during reward-related decision making. , 2009, Cerebral cortex.

[55]  Viktor Müller,et al.  Life Span Differences in Electrophysiological Correlates of Monitoring Gains and Losses during Probabilistic Reinforcement Learning , 2011, Journal of Cognitive Neuroscience.

[56]  P. Dayan,et al.  States versus Rewards: Dissociable Neural Prediction Error Signals Underlying Model-Based and Model-Free Reinforcement Learning , 2010, Neuron.

[57]  S. Petersen,et al.  Role of the anterior insula in task-level control and focal attention , 2010, Brain Structure and Function.

[58]  H. Critchley,et al.  A common role of insula in feelings, empathy and uncertainty , 2009, Trends in Cognitive Sciences.

[59]  M. Hallett,et al.  Mechanisms Underlying Dopamine-Mediated Reward Bias in Compulsive Behaviors , 2010, Neuron.

[60]  D. E. Goldberg,et al.  Genetic Algorithms in Search , 1989 .

[61]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[62]  E. Leibenluft,et al.  Developmental effects of decision-making on sensitivity to reward: An fMRI study , 2012, Developmental Cognitive Neuroscience.

[63]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[64]  K. Zanolie,et al.  Neural mechanisms supporting flexible performance adjustment during development , 2008, Cognitive, affective & behavioral neuroscience.

[65]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[66]  T. Paus,et al.  Why do many psychiatric disorders emerge during adolescence? , 2008, Nature Reviews Neuroscience.

[67]  R. Dolan,et al.  Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans , 2006, Nature.

[68]  T. Robbins,et al.  Decision-making in the adolescent brain , 2012, Nature Neuroscience.

[69]  S. Hofmann Perception of control over anxiety mediates the relation between catastrophic thinking and social anxiety in social phobia. , 2005, Behaviour research and therapy.

[70]  M. Brammer,et al.  Progressive increase of frontostriatal brain activation from childhood to adulthood during event‐related tasks of cognitive control , 2006, Human brain mapping.

[71]  Karl J. Friston,et al.  Bayesian model selection for group studies , 2009, NeuroImage.

[72]  Corianne Rogalsky,et al.  Increased activation in the right insula during risk-taking decision making is related to harm avoidance and neuroticism , 2003, NeuroImage.

[73]  Zhong-Lin Lu,et al.  Common Neural Mechanisms Underlying Reversal Learning by Reward and Punishment , 2013, PloS one.

[74]  Dirk H. K. Schmidt,et al.  Differential representation of feedback and decision in adolescents and adults , 2014, Neuropsychologia.

[75]  L. Somerville The Teenage Brain , 2013 .

[76]  N. Daw,et al.  Striatal Activity Underlies Novelty-Based Choice in Humans , 2008, Neuron.

[77]  M. Frank,et al.  Frontal theta as a mechanism for cognitive control , 2014, Trends in Cognitive Sciences.

[78]  J. O'Doherty,et al.  Decoding the neural substrates of reward-related decision making with functional MRI , 2007, Proceedings of the National Academy of Sciences.

[79]  P. Dayan,et al.  Serotonin Selectively Modulates Reward Value in Human Decision-Making , 2012, The Journal of Neuroscience.

[80]  Peter Stiers,et al.  Developmental changes between ages 13 and 21years in the extent and magnitude of the BOLD response during decision making , 2011, NeuroImage.

[81]  Mick Brammer,et al.  Sex-dependent age modulation of frontostriatal and temporo-parietal activation during cognitive control , 2009, NeuroImage.

[82]  Daniel Brandeis,et al.  Role of the Medial Prefrontal Cortex in Impaired Decision Making in Juvenile Attention-Deficit/Hyperactivity Disorder , 2014 .

[83]  Carrie L. Masten,et al.  Neural correlates of social exclusion during adolescence: understanding the distress of peer rejection. , 2009, Social cognitive and affective neuroscience.

[84]  Russell A. Poldrack,et al.  A unique adolescent response to reward prediction errors , 2010, Nature Neuroscience.

[85]  R. Dolan,et al.  Dopamine, Time, and Impulsivity in Humans , 2010, The Journal of Neuroscience.

[86]  W. Schultz Getting Formal with Dopamine and Reward , 2002, Neuron.

[87]  Jan Gläscher,et al.  Visualization of Group Inference Data in Functional Neuroimaging , 2009, Neuroinformatics.

[88]  Carter Wendelken,et al.  Flexible rule use: Common neural substrates in children and adults , 2012, Developmental Cognitive Neuroscience.

[89]  Toshio Iijima,et al.  Inactivating Anterior Insular Cortex Reduces Risk Taking , 2012, The Journal of Neuroscience.