Learning abstract structure for drawing by efficient motor program induction

Humans flexibly solve new problems that differ qualitatively from those they were trained on. This ability to generalize is supported by learned concepts that capture structure common across different problems. Here we develop a naturalistic drawing task to study how humans rapidly acquire structured prior knowledge. The task requires drawing visual objects that share underlying structure, based on a set of composable geometric rules. We show that people spontaneously learn abstract drawing procedures that support generalization, and propose a model of how learners can discover these reusable drawing programs. Trained in the same setting as humans, and constrained to produce efficient motor actions, this model discovers new drawing routines that transfer to test objects and resemble learned features of human sequences. These results suggest that two principles guiding motor program induction in the model - abstraction (general programs that ignore object-specific details) and compositionality (recombining previously learned programs) - are key for explaining how humans learn structured internal representations that guide flexible reasoning and learning.

[1]  Charles Kemp,et al.  How to Grow a Mind: Statistics, Structure, and Abstraction , 2011, Science.

[2]  F. Bartlett,et al.  Remembering: A Study in Experimental and Social Psychology , 1932 .

[3]  Charles Kemp,et al.  The discovery of structural form , 2008, Proceedings of the National Academy of Sciences.

[4]  Jimei Yang,et al.  Learning to Doodle with Stroke Demonstrations and Deep Q-Networks , 2018, BMVC.

[5]  Brenden M. Lake,et al.  Compositional generalization through meta sequence-to-sequence learning , 2019, NeurIPS.

[6]  Steve Piantadosi,et al.  Knowledge transfer in a probabilistic Language Of Thought , 2017, CogSci.

[7]  Armando Solar-Lezama,et al.  DreamCoder: growing generalizable, interpretable knowledge with wake–sleep Bayesian program learning , 2020, Philosophical Transactions of the Royal Society A.

[8]  S. Carey.  The Origin of Concepts , 2000 .

[9]  Kenneth D. Forbus,et al.  CogSketch: Sketch Understanding for Cognitive Science Research and for Education , 2011, Top. Cogn. Sci..

[10]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[11]  S. Dehaene,et al.  Bayesian selection of grammar productions for the language of thought , 2017, bioRxiv.

[12]  Chuang Gan,et al.  The Neuro-Symbolic Concept Learner: Interpreting Scenes Words and Sentences from Natural Supervision , 2019, ICLR.

[13]  Nando de Freitas,et al.  Neural Programmer-Interpreters , 2015, ICLR.

[14]  S. Kay.  On the Nature of Expertise. , 1992 .

[15]  Jaakko Hintikka,et al.  On the Logic of Perception , 1969 .

[16]  D. McDermott.  LANGUAGE OF THOUGHT , 2012 .

[17]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[18]  J. Tenenbaum,et al.  Bayesian Special Section Learning Overhypotheses with Hierarchical Bayesian Models , 2022 .

[19]  Jessica B. Hamrick,et al.  psiTurk: An open-source framework for conducting replicable behavioral experiments online , 2016, Behavior research methods.

[20]  Elizabeth F. Pemberton.  The drawing rules of children: Sequence and direction , 1987 .

[21]  Joshua B. Tenenbaum,et al.  Efficient inverse graphics in biological face processing , 2018, Science Advances.

[22]  Armando Solar-Lezama,et al.  Learning Libraries of Subroutines for Neurally-Guided Bayesian Program Induction , 2018, NeurIPS.

[23]  Sergio Gomez Colmenarejo,et al.  Hybrid computing using a neural network with dynamic external memory , 2016, Nature.

[24]  Michael C. Frank,et al.  Drawings as a window into the development of object category representations , 2018 .

[25]  Ulrich Eggers,et al.  Drawing And Cognition Descriptive And Experimental Studies Of Graphic Production Processes , 2016 .

[26]  Joshua B. Tenenbaum,et al.  Grammar-based object representations in a scene parsing task , 2009 .

[27]  J. Fodor,et al.  Connectionism and cognitive architecture: A critical analysis , 1988, Cognition.

[28]  Steve Piantadosi,et al.  People Infer Recursive Visual Concepts from Just a Few Examples , 2019, Computational Brain & Behavior.

[29]  Peter C.-H. Cheng,et al.  Drawing out the Temporal Signature of Induced Perceptual Chunks , 2001 .

[30]  Jacob Feldman,et al.  The simplicity principle in perception and cognition. , 2016, Wiley interdisciplinary reviews. Cognitive science.

[31]  Alex Graves,et al.  DRAW: A Recurrent Neural Network For Image Generation , 2015, ICML.

[32]  Dileep George,et al.  Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programs , 2018, Science Robotics.

[33]  Elizabeth S Spelke,et al.  Core Knowledge and the Emergence of Symbols: The Case of Maps , 2015, Journal of cognition and development : official journal of the Cognitive Development Society.

[34]  Robert A. Jacobs,et al.  From Sensory Signals to Modality-Independent Conceptual Representations: A Probabilistic Language of Thought Approach , 2015, PLoS Comput. Biol..

[35]  Armando Solar-Lezama,et al.  Learning to Infer Graphics Programs from Hand-Drawn Images , 2017, NeurIPS.

[36]  Elizabeth S. Spelke,et al.  Core Knowledge, Language, and Number , 2017 .

[37]  Yoshua Bengio,et al.  Drawing and Recognizing Chinese Characters with Recurrent Neural Network , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  H. Harlow,et al.  The formation of learning sets. , 1949, Psychological review.

[39]  Eunbyung Park,et al.  Unsupervised Doodling and Painting with Improved SPIRAL , 2019, ArXiv.

[40]  Armando Solar-Lezama,et al.  Library learning for neurally-guided Bayesian program induction , 2018, NIPS 2018.

[41]  Daniel L. K. Yamins,et al.  Common Object Representations for Visual Production and Recognition , 2018, Cogn. Sci..

[42]  Norman H. Freeman,et al.  Visual order : the nature and development of pictorial representation , 1985 .

[43]  Sumit Gulwani,et al.  Automating string processing in spreadsheets using input-output examples , 2011, POPL '11.

[44]  Thomas L. Griffiths,et al.  A Rational Analysis of Rule-Based Concept Learning , 2008, Cogn. Sci..

[45]  Douglas Eck,et al.  A Neural Representation of Sketch Drawings , 2017, ICLR.

[46]  Joshua B. Tenenbaum,et al.  Learning list concepts through program induction , 2018, bioRxiv.

[47]  Timothy J. O'Donnell,et al.  Compositional Policy Priors , 2013 .

[48]  A. Karmiloff-Smith.  Constraints on representational change: Evidence from children's drawing , 1990, Cognition.