DreamCoder: growing generalizable, interpretable knowledge with wake–sleep Bayesian program learning

Expert problem-solving is driven by powerful languages for thinking about problems and their solutions. Acquiring expertise means learning these languages -- systems of concepts, alongside the skills to use them. We present DreamCoder, a system that learns to solve problems by writing programs. It builds expertise by creating programming languages for expressing domain concepts, together with neural networks to guide the search for programs within these languages. A ``wake-sleep'' learning algorithm alternately extends the language with new symbolic abstractions and trains the neural network on imagined and replayed problems. DreamCoder solves both classic inductive programming tasks and creative tasks such as drawing pictures and building scenes. It rediscovers the basics of modern functional programming, vector algebra and classical physics, including Newton's and Coulomb's laws. Concepts are built compositionally from those learned earlier, yielding multi-layered symbolic representations that are interpretable and transferrable to new tasks, while still growing scalably and flexibly with experience.

[1]  Marcus Hutter.  Simulation Algorithms for Computational Systems Biology , 2017, Texts in Theoretical Computer Science. An EATCS Series.

[2]  Matthew J. Hausknecht,et al.  Neural Program Meta-Induction , 2017, NIPS.

[3]  Rajeev Alur,et al.  SyGuS-Comp 2017: Results and Analysis , 2017, SYNT@CAV.

[4]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[5]  D. McDermott.  LANGUAGE OF THOUGHT , 2012 .

[6]  Hod Lipson,et al.  Distilling Free-Form Natural Laws from Experimental Data , 2009, Science.

[7]  Paul J. Feltovich,et al.  Categorization and Representation of Physics Problems by Experts and Novices , 1981, Cogn. Sci..

[8]  S. Kay.  On the Nature of Expertise. , 1992 .

[9]  Tom M. Mitchell,et al.  Version Spaces: A Candidate Elimination Approach to Rule Learning , 1977, IJCAI.

[10]  Peter M. Vishton,et al.  Rule learning by seven-month-old infants. , 1999, Science.

[11]  Dan Klein,et al.  Neural Module Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Reuben Feinman,et al.  Generating new concepts with hybrid neuro-symbolic models , 2020, CogSci.

[13]  Martin Stacey,et al.  Scientific Discovery: Computational Explorations of the Creative Processes , 1988 .

[14]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[15]  Dan Klein,et al.  Learning Dependency-Based Compositional Semantics , 2011, CL.

[16]  Pushmeet Kohli,et al.  RobustFill: Neural Program Learning under Noisy I/O , 2017, ICML.

[17]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part I , 1964, Inf. Control..

[18]  J. Hobson,et al.  Dreaming and Episodic Memory: A Functional Dissociation? , 2003, Journal of Cognitive Neuroscience.

[19]  Jürgen Schmidhuber,et al.  Optimal Ordered Problem Solver , 2002, Machine Learning.

[20]  Wojciech Zaremba,et al.  Domain randomization for transferring deep neural networks from simulation to the real world , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[21]  R. Solomonoff.  A SYSTEM FOR INCREMENTAL LEARNING BASED ON ALGORITHMIC PROBABILITY , 1989 .

[22]  Nick Chater,et al.  Programs as Causal Models: Speculations on Mental Programs and Mental Representation , 2013, Cogn. Sci..

[23]  Mayur Naik,et al.  Learning Neurosymbolic Generative Models via Program Synthesis , 2019, ICML.

[24]  Geoffrey E. Hinton,et al.  The "wake-sleep" algorithm for unsupervised neural networks. , 1995, Science.

[25]  Joshua B. Tenenbaum,et al.  Building machines that learn and think like people , 2016, Behavioral and Brain Sciences.

[26]  Jonathan Evans.  Heuristic and analytic processes in reasoning , 1984 .

[27]  H. Simon,et al.  Scientific discovery as problem solving , 1981, Synthese.

[28]  J. Raven,et al.  Raven Progressive Matrices , 2003 .

[29]  Sergio Gomez Colmenarejo,et al.  Hybrid computing using a neural network with dynamic external memory , 2016, Nature.

[30]  Sumit Gulwani,et al.  FlashMeta: a framework for inductive program synthesis , 2015, OOPSLA.

[31]  Joshua B. Tenenbaum,et al.  Human-level concept learning through probabilistic program induction , 2015, Science.

[32]  Adam Wierman,et al.  Thinking Fast and Slow , 2017, SIGMETRICS Perform. Evaluation Rev..

[33]  Sebastian Nowozin,et al.  DeepCoder: Learning to Write Programs , 2016, ICLR.

[34]  Jason Weston,et al.  Curriculum learning , 2009, ICML '09.

[35]  Isil Dillig,et al.  Synthesizing data structure transformations from input-output examples , 2015, PLDI.

[36]  Noah D. Goodman.  Learning and the language of thought , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[37]  Pedro M. Domingos,et al.  Programming by Demonstration Using Version Space Algebra , 2003, Machine Learning.

[38]  Armando Solar-Lezama,et al.  Learning to Infer Graphics Programs from Hand-Drawn Images , 2017, NeurIPS.

[39]  Armando Solar-Lezama,et al.  Program synthesis by sketching , 2008 .

[40]  Swarat Chaudhuri,et al.  HOUDINI: Lifelong Learning as Program Synthesis , 2018, NeurIPS.

[41]  S. Carey.  The Origin of Concepts : A Precis Susan Carey Harvard University , 2012 .

[42]  Andrew Cropper,et al.  Playgol: learning programs through play , 2019, IJCAI.

[43]  Ray J. Solomonoff,et al.  A Formal Theory of Inductive Inference. Part II , 1964, Inf. Control..

[44]  Michael I. Jordan,et al.  Learning Programs: A Hierarchical Bayesian Approach , 2010, ICML.

[45]  Joshua B. Tenenbaum,et al.  Bootstrap Learning via Modular Concept Discovery , 2013, IJCAI.

[46]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[47]  E. Spelke,et al.  Origins of knowledge. , 1992, Psychological review.

[48]  Ray J. Solomonofi,et al.  A SYSTEM FOR INCREMENTAL LEARNING BASED ON ALGORITHMIC PROBABILITY , 1989 .

[49]  J. Elman.  Learning and development in neural networks: the importance of starting small , 1993, Cognition.

[50]  Armando Solar-Lezama,et al.  Library learning for neurally-guided Bayesian program induction , 2018, NIPS 2018.

[51]  Yadin Dudai,et al.  The Consolidation and Transformation of Memory , 2015, Neuron.

[52]  D. Hofstadter,et al.  Letter Spirit: an Emergent Model of the Perception and Creation of Alphabetic Style , 1993 .

[53]  Sumit Gulwani,et al.  Automating string processing in spreadsheets using input-output examples , 2011, POPL '11.

[54]  Michael Stepp,et al.  Equality saturation: a new approach to optimization , 2009, POPL '09.

[55]  Luc De Raedt,et al.  DeepProbLog: Neural Probabilistic Logic Programming , 2018, BNAIC/BENELEARN.

[56]  Dileep George,et al.  Beyond imitation: Zero-shot task transfer on robots by learning concepts as cognitive programs , 2018, Science Robotics.

[57]  Joshua B. Tenenbaum,et al.  Picture: A probabilistic programming language for scene perception , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  James R. Glass,et al.  One-shot learning of generative speech concepts , 2014, CogSci.