Erasing the engram: the unlearning of procedural skills.

Huge amounts of money are spent every year on unlearning programs--in drug-treatment facilities, prisons, psychotherapy clinics, and schools. Yet almost all of these programs fail, since recidivism rates are high in each of these fields. Progress on this problem requires a better understanding of the mechanisms that make unlearning so difficult. Much cognitive neuroscience evidence suggests that an important component of these mechanisms also dictates success on categorization tasks that recruit procedural learning and depend on synaptic plasticity within the striatum. A biologically detailed computational model of this striatal-dependent learning is described (based on Ashby & Crossley, 2011). The model assumes that a key component of striatal-dependent learning is provided by interneurons in the striatum called the tonically active neurons (TANs), which act as a gate for the learning and expression of striatal-dependent behaviors. In their tonically active state, the TANs prevent the expression of any striatal-dependent behavior. However, they learn to pause in rewarding environments and thereby permit the learning and expression of striatal-dependent behaviors. The model predicts that when rewards are no longer contingent on behavior, the TANs cease to pause, which protects striatal learning from decay and prevents unlearning. In addition, the model predicts that when rewards are partially contingent on behavior, the TANs remain partially paused, leaving the striatum available for unlearning. The results from 3 human behavioral studies support the model predictions and suggest a novel unlearning protocol that shows promising initial signs of success.

[1]  S. Klein,et al.  Handbook of contemporary learning theories , 2000 .

[2]  E. T. Rolls,et al.  Responses of striatal neurons in the behaving monkey. 3. Effects of iontophoretically applied dopamine on normal responsiveness , 1984, Neuroscience.

[3]  Shawn W. Ell,et al.  Prefrontal Contributions to Rule-based and Information-integration Category Learning , 2022 .

[4]  M. Bouton,et al.  A retrieval cue for extinction attenuates response recovery (renewal) caused by a return to the conditioning context. , 1994 .

[5]  N. Petry A comprehensive guide to the application of contingency management procedures in clinical settings. , 2000, Drug and alcohol dependence.

[6]  M. Bouton Context and behavioral processes in extinction. , 2004, Learning & memory.

[7]  A. Graybiel,et al.  Neurons in the thalamic CM-Pf complex supply striatal neurons with information about behaviorally significant sensory events. , 2001, Journal of neurophysiology.

[8]  W. Schultz,et al.  Coding of Predicted Reward Omission by Dopamine Neurons in a Conditioned Inhibition Paradigm , 2003, The Journal of Neuroscience.

[9]  Gregory Ashby,et al.  A neuropsychological theory of multiple systems in category learning. , 1998, Psychological review.

[10]  R. Nosofsky Attention, similarity, and the identification-categorization relationship. , 1986, Journal of experimental psychology. General.

[11]  Matthew J. Crossley,et al.  Interactions between declarative and procedural-learning categorization systems , 2010, Neurobiology of Learning and Memory.

[12]  Corey J Bohil,et al.  Observational versus feedback training in rule-based and information-integration category learning , 2002, Memory & cognition.

[13]  M. Bouton Context, ambiguity, and unlearning: sources of relapse after behavioral extinction , 2002, Biological Psychiatry.

[14]  E. T. Rolls,et al.  Responses of striatal neurons in the behaving monkey. 2. Visual processing in the caudal neostriatum , 1984, Brain Research.

[15]  H. E. Rosvold,et al.  Behavioral effects of selective ablation of the caudate nucleus. , 1967, Journal of comparative and physiological psychology.

[16]  S. Lewandowsky,et al.  Knowledge partitioning in categorization: constraints on exemplar models. , 2004, Journal of experimental psychology. Learning, memory, and cognition.

[17]  Donald Hedeker,et al.  Estimation of Effect Size From a Series of Experiments Involving Paired Comparisons , 1993 .

[18]  W. T. Maddox,et al.  Dual-task interference in perceptual category learning , 2006, Memory & cognition.

[19]  M. Bouton,et al.  A retrieval cue for extinction attenuates spontaneous recovery. , 1993, Journal of experimental psychology. Animal behavior processes.

[20]  P. Calabresi,et al.  The corticostriatal projection: from synaptic plasticity to dysfunctions of the basal ganglia , 1996, Trends in Neurosciences.

[21]  D H BULLOCK,et al.  An effect of repeated conditioning-extinction upon operant strength. , 1953, Journal of experimental psychology.

[22]  W. Estes Array models for category learning , 1986, Cognitive Psychology.

[23]  O. Burešová,et al.  Effect of atropine on learning, extinction, retention and retrieval in rats , 1964, Psychopharmacologia.

[24]  W Todd Maddox,et al.  Information-integration category learning in patients with striatal dysfunction. , 2005, Neuropsychology.

[25]  Carol A. Seger,et al.  The Roles of the Caudate Nucleus in Human Classification Learning , 2005, The Journal of Neuroscience.

[26]  Douglas L. Hintzman,et al.  "Schema Abstraction" in a Multiple-Trace Memory Model , 1986 .

[27]  W. T. Maddox,et al.  Delayed feedback disrupts the procedural-learning system but not the hypothesis-testing system in perceptual category learning. , 2005, Journal of experimental psychology. Learning, memory, and cognition.

[28]  W Todd Maddox,et al.  Disrupting feedback processing interferes with rule-based but not information-integration category learning , 2004, Memory & cognition.

[29]  J. Pearce,et al.  A model for Pavlovian learning: Variations in the effectiveness of conditioned but not of unconditioned stimuli. , 1980 .

[30]  F. Gregory Ashby,et al.  Categorization response time with multidimensional stimuli , 1994, Perception & psychophysics.

[31]  R. R. Bush,et al.  A Mathematical Model for Simple Learning , 1951 .

[32]  W. T. Maddox,et al.  Neural correlates of rule-based and information-integration visual category learning. , 2006, Cerebral cortex.

[33]  W. Rall Distinguishing theoretical synaptic potentials computed for different soma-dendritic distributions of synaptic input. , 1967, Journal of neurophysiology.

[34]  Karl J. Friston,et al.  Dissociable Roles of Ventral and Dorsal Striatum in Instrumental Conditioning , 2004, Science.

[35]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[36]  John M. Ennis,et al.  A neurobiological theory of automaticity in perceptual categorization. , 2007, Psychological review.

[37]  J. Kruschke,et al.  ALCOVE: an exemplar-based connectionist model of category learning. , 1992, Psychological review.

[38]  F. Ashby,et al.  On the nature of implicit categorization , 1999, Psychonomic bulletin & review.

[39]  M. Jarvik,et al.  Contingency management for tobacco smoking in methadone-maintained opiate addicts. , 1996, Addictive behaviors.

[40]  F. Gregory Ashby,et al.  Cortical and striatal contributions to automaticity in information-integration categorization , 2011, NeuroImage.

[41]  W. T. Maddox,et al.  Stimulus range and discontinuity effects on information-integration category learning and generalization , 2011, Attention, perception & psychophysics.

[42]  E. Fischer Conditioned Reflexes , 1942, American journal of physical medicine.

[43]  David Marr,et al.  VISION A Computational Investigation into the Human Representation and Processing of Visual Information , 2009 .

[44]  Adam N Sanborn,et al.  Rational approximations to rational models: alternative algorithms for category learning. , 2010, Psychological review.

[45]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[46]  W T Maddox,et al.  Comparing decision bound and exemplar models of categorization , 1993, Perception & psychophysics.

[47]  Bard Ermentrout,et al.  Type I Membranes, Phase Resetting Curves, and Synchrony , 1996, Neural Computation.

[48]  Peter Redgrave,et al.  Basal Ganglia , 2020, Encyclopedia of Autism Spectrum Disorders.

[49]  F Gregory Ashby,et al.  Analogical transfer in perceptual categorization , 2011, Memory & Cognition.

[50]  Douglas L. Medin,et al.  Context theory of classification learning. , 1978 .

[51]  F. Ashby,et al.  Category learning and multiple memory systems , 2005, Trends in Cognitive Sciences.

[52]  Shawn W. Ell,et al.  Procedural learning in perceptual categorization , 2003, Memory & cognition.

[53]  Corey J. Bohil,et al.  Delayed feedback effects on rule-based and information-integration category learning. , 2003, Journal of experimental psychology. Learning, memory, and cognition.

[54]  Gregory Ashby,et al.  Decision rules in the perception and categorization of multidimensional stimuli. , 1988, Journal of experimental psychology. Learning, memory, and cognition.

[55]  J. Kruschke Models of Attentional Learning , 2011 .

[56]  R. Bolles,et al.  Contextual control of the extinction of conditioned fear , 1979 .

[57]  Hisashi Q. Higuchi On the nature of , 1999 .

[58]  Michelle R. Molis,et al.  Generalizing a neuropsychological model of visual categorization to auditory categorization of vowels , 2002, Perception & psychophysics.

[59]  R. Rescorla,et al.  Effect of response-independent reinforcers during extinction. , 1969 .

[60]  J. Kruschke,et al.  Rules and exemplars in category learning. , 1998, Journal of experimental psychology. General.

[61]  K. Urushihara,et al.  Renewal of Extinguished Lever-Press Responses upon Return to the Training Context , 2000 .

[62]  Gregory Ashby,et al.  Suboptimality in human categorization and identification. , 2001, Journal of experimental psychology. General.

[63]  D. A. King,et al.  Contextual control of the extinction of conditioned fear: tests for the associative value of the context. , 1983, Journal of experimental psychology. Animal behavior processes.

[64]  Corey J. Bohil,et al.  Evidence for a procedural-learning-based system in perceptual category learning , 2004, Psychonomic bulletin & review.

[65]  R. Boakes Response decrements produced by extinction and by response-independent reinforcement. , 1973, Journal of the experimental analysis of behavior.

[66]  T. Poggio,et al.  Hierarchical models of object recognition in cortex , 1999, Nature Neuroscience.

[67]  David T. Neal,et al.  A new look at habits and the habit-goal interface. , 2007, Psychological review.

[68]  F. Ashby,et al.  The effects of concurrent task interference on category learning: Evidence for multiple category learning systems , 2001, Psychonomic bulletin & review.

[69]  F. Gregory Ashby,et al.  A Computational Model of How Cholinergic Interneurons Protect Striatal-dependent Learning , 2011, Journal of Cognitive Neuroscience.

[70]  P. Glimcher,et al.  Midbrain Dopamine Neurons Encode a Quantitative Reward Prediction Error Signal , 2005, Neuron.

[71]  J. Oglesby,et al.  Radial basis function networks for speaker recognition , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[72]  Meng Joo Er,et al.  Face recognition with radial basis function (RBF) neural networks , 2002, IEEE Trans. Neural Networks.

[73]  E. Rosch,et al.  Cognition and Categorization , 1980 .

[74]  Jadin C. Jackson,et al.  Reconciling reinforcement learning models with behavioral extinction and renewal: implications for addiction, relapse, and problem gambling. , 2007, Psychological review.

[75]  Joshua W. Brown,et al.  How the Basal Ganglia Use Parallel Excitatory and Inhibitory Learning Pathways to Selectively Respond to Unexpected Rewarding Cues , 1999, The Journal of Neuroscience.

[76]  William K. Estes,et al.  Classification and cognition , 1994 .

[77]  F. Gregory Ashby,et al.  Multidimensional Models of Perception and Cognition , 2014 .

[78]  W. Estes Statistical theory of spontaneous recovery and regression. , 1955, Psychological review.

[79]  Larry S. Davis,et al.  Human expression recognition from motion using a radial basis function network architecture , 1996, IEEE Trans. Neural Networks.

[80]  James L. McClelland,et al.  The time course of perceptual choice: the leaky, competing accumulator model. , 2001, Psychological review.

[81]  W. Bickel,et al.  Contingent reinforcement of abstinence with individuals abusing cocaine and marijuana. , 1991, Journal of applied behavior analysis.

[82]  A. Graybiel,et al.  Activity of striatal neurons reflects dynamic encoding and recoding of procedural memories , 2005, Nature.

[83]  F. Gregory Ashby,et al.  THE ROLE OF THE BASAL GANGLIA IN CATEGORY LEARNING , 2006 .

[84]  K. Urushihara,et al.  Renewal of operant performance formerly eliminated by omission or noncontingency training upon return to the acquisition context , 2002 .

[85]  van der Steen The timing of movements and the role of the basal ganglia , 2013 .

[86]  W. Schultz Midbrain Dopamine Neurons , 2009 .

[87]  B. Balleine,et al.  The role of the dorsomedial striatum in instrumental conditioning , 2005, The European journal of neuroscience.

[88]  R. O’Reilly,et al.  Computational Explorations in Cognitive Neuroscience , 2009 .

[89]  M. Bear,et al.  Experience-dependent modification of synaptic plasticity in visual cortex , 1996, Nature.

[90]  W. Todd Maddox,et al.  Rule-based and information-integration category learning in normal aging , 2010, Neuropsychologia.

[91]  Brian D. Glass,et al.  When more is less: Feedback effects in perceptual category learning , 2008, Cognition.

[92]  F. Gregory Ashby,et al.  Multidimensional models of categorization. , 1992 .

[93]  David L. Faigman,et al.  Human category learning. , 2005, Annual review of psychology.

[94]  José M. R. Delgado,et al.  Integrative Activity of the Brain , 1968, The Yale Journal of Biology and Medicine.

[95]  Brian D. Glass,et al.  Category label and response location shifts in category learning , 2009, Psychological research.

[96]  E. Bienenstock,et al.  Theory for the development of neuron selectivity: orientation specificity and binocular interaction in visual cortex , 1982, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[97]  Jennifer A. Mangels,et al.  A Neostriatal Habit Learning System in Humans , 1996, Science.

[98]  B. Balleine,et al.  Goal-directed instrumental action: contingency and incentive learning and their cortical substrates , 1998, Neuropharmacology.

[99]  M. Bouton,et al.  Contextual and Temporal Modulation of Extinction: Behavioral and Biological Mechanisms , 2006, Biological Psychiatry.

[100]  R. R. Miller,et al.  Conducting exposure treatment in multiple contexts can prevent relapse. , 1998, Behaviour research and therapy.

[101]  Eugene M. Izhikevich,et al.  Dynamical Systems in Neuroscience: The Geometry of Excitability and Bursting , 2006 .

[102]  D. Blei,et al.  Context, learning, and extinction. , 2010, Psychological review.

[103]  M. Kawato,et al.  Different neural correlates of reward expectation and reward expectation error in the putamen and caudate nucleus during stimulus-action-reward association learning. , 2006, Journal of neurophysiology.

[104]  H. Barlow Vision: A computational investigation into the human representation and processing of visual information: David Marr. San Francisco: W. H. Freeman, 1982. pp. xvi + 397 , 1983 .

[105]  J. Konorski Conditioned reflexes and neuron organization. , 1948 .

[106]  G. Badger,et al.  Outpatient behavioral treatment for cocaine dependence: One-year outcome. , 1995 .

[107]  M. Bouton,et al.  Occasional Reinforced Responses During Extinction Can Slow the Rate of Reacquisition of an Operant Response. , 2007, Learning and motivation.

[108]  K. Lamberts Information-accumulation theory of speeded categorization. , 2000, Psychological review.