Learning and transfer of working memory gating policies

Abstract knowledge about the tasks we encounter enables us to rapidly and flexibly adapt to novel task contexts. Previous research has focused primarily on abstract rules that leverage shared structure in stimulus-response (S-R) mappings as the basis of such task knowledge. Here we provide evidence that working memory (WM) gating policies – a type of control policy required for internal control of WM during a task – constitute a form of abstract task knowledge that can be transferred across contexts. In two experiments, we report specific evidence for the transfer of selective WM gating policies across changes of task context. We show that this transfer is not tied to shared structure in S-R mappings, but instead in the dynamic structure of the task. Collectively, our results highlight the importance of WM gating policies in particular, and control policies in general, as a key component of the task knowledge that supports flexible behavior and task generalization.

[1]  Charles Kemp,et al.  How to Grow a Mind: Statistics, Structure, and Abstraction , 2011, Science.

[2]  Thomas E. Hazy,et al.  Towards an executive without a homunculus: computational models of the prefrontal cortex/basal ganglia system , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[3]  D. Blei,et al.  Context, learning, and extinction. , 2010, Psychological review.

[4]  T. Braver.  The variable nature of cognitive control: a dual mechanisms framework , 2012, Trends in Cognitive Sciences.

[5]  K. Oberauer,et al.  Attention to Information in Working Memory , 2012 .

[6]  Robert A. Jacobs,et al.  The Adaptive Nature of Visual Working Memory , 2014 .

[7]  Peter Dayan,et al.  Interplay of approximate planning strategies , 2015, Proceedings of the National Academy of Sciences.

[8]  N. Taatgen.  The nature and transfer of cognitive skills. , 2013, Psychological review.

[9]  G. A. Miller.  THE PSYCHOLOGICAL REVIEW THE MAGICAL NUMBER SEVEN, PLUS OR MINUS TWO: SOME LIMITS ON OUR CAPACITY FOR PROCESSING INFORMATION 1 , 1956 .

[10]  Michael J. Frank,et al.  Making Working Memory Work: A Computational Model of Learning in the Prefrontal Cortex and Basal Ganglia , 2006, Neural Computation.

[11]  Jonathan D. Cohen,et al.  Learning to Use Working Memory in Partially Observable Environments through Dopaminergic Reinforcement , 2008, NIPS.

[12]  Jonathan D. Cohen,et al.  Prefrontal cortex and flexible cognitive control: rules without symbols. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[13]  Anne G E Collins,et al.  Cognitive control over learning: creating, clustering, and generalizing task-set structure. , 2013, Psychological review.

[14]  Roger C. Schank,et al.  Scripts, plans, goals and understanding: an inquiry into human knowledge structures , 1978 .

[15]  Y. Niv,et al.  Learning latent structure: carving nature at its joints , 2010, Current Opinion in Neurobiology.

[16]  Christopher H Chatham,et al.  Multiple gates on working memory , 2015, Current Opinion in Behavioral Sciences.

[17]  Michael W. Cole,et al.  The power of instructions: Proactive configuration of stimulus-response translation. , 2015, Journal of experimental psychology. Learning, memory, and cognition.

[18]  Hannes Ruge,et al.  Rapid formation of pragmatic rule representations in the human brain during instruction-based learning. , 2010, Cerebral cortex.

[19]  F. Mathy,et al.  What’s magic about magic numbers? Chunking and data compression in short-term memory , 2012, Cognition.

[20]  Herbert A. Simon,et al.  The Structure of Ill Structured Problems , 1973, Artif. Intell..

[21]  Michael W. Cole,et al.  Prefrontal Dynamics Underlying Rapid Instructed Task Learning Reverse with Practice , 2010, The Journal of Neuroscience.

[22]  John Duncan,et al.  Goal neglect and knowledge chunking in the construction of novel behaviour☆ , 2014, Cognition.

[23]  M. Frank,et al.  Mechanisms of hierarchical reinforcement learning in corticostriatal circuits 1: computational analysis. , 2012, Cerebral cortex.

[24]  Hannes Ruge,et al.  On the timescale of stimulus-based action–effect learning , 2011, Quarterly journal of experimental psychology.

[25]  Daniel A. Braun,et al.  Structure learning in action , 2010, Behavioural Brain Research.

[26]  James L. McClelland.  Is a Machine Realization of Truly Human-Like Intelligence Achievable? , 2009, Cognitive Computation.

[27]  Timothy F. Brady,et al.  Compression in visual working memory: using statistical regularities to form more efficient memory representations. , 2009, Journal of experimental psychology. General.

[28]  Alec Solway,et al.  Reinforcement learning, efficient coding, and the statistics of natural tasks , 2015, Current Opinion in Behavioral Sciences.

[29]  Daniel A. Braun,et al.  Motor Task Variation Induces Structural Learning , 2009, Current Biology.

[30]  E. Koechlin,et al.  Reasoning, Learning, and Creativity: Frontal Lobe Function and Human Decision-Making , 2012, PLoS biology.

[31]  D. Shanks,et al.  FEATURE- AND RULE-BASED GENERALIZATION IN HUMAN ASSOCIATIVE LEARNING , 1998 .

[32]  Christopher H. Chatham,et al.  Corticostriatal Output Gating during Selection from Working Memory , 2014, Neuron.

[33]  Nachshon Meiran,et al.  The representation of instructions in working memory leads to autonomous response activation: Evidence from the first trials in the flanker paradigm , 2006, Quarterly journal of experimental psychology.

[34]  P. Ackerman.  Determinants of individual differences during skill acquisition: Cognitive abilities and information processing. , 1988 .

[35]  Michael W. Cole,et al.  Rapid Transfer of Abstract Rules to Novel Contexts in Human Lateral Prefrontal Cortex , 2011, Front. Hum. Neurosci..

[36]  Tobias Egner,et al.  The Wiley Handbook of Cognitive Control , 2017 .

[37]  Nachshon Meiran,et al.  The representation of instructions operates like a prepared reflex: flanker compatibility effects found in first trial following S-R instructions. , 2009, Experimental psychology.

[38]  John Duncan,et al.  Goal neglect and Spearman's g: competing parts of a complex task. , 2008, Journal of experimental psychology. General.

[39]  M. Botvinick,et al.  Hierarchically organized behavior and its neural foundations: A reinforcement learning perspective , 2009, Cognition.

[40]  Jonathan D. Cohen,et al.  Indirection and symbol-like processing in the prefrontal cortex and basal ganglia , 2013, Proceedings of the National Academy of Sciences.

[41]  M. D’Esposito,et al.  Frontal Cortex and the Discovery of Abstract Action Rules , 2010, Neuron.

[42]  Harold Bekkering,et al.  Dissociating restart cost and mixing cost in task switching , 2009, Psychological research.