Learning Neural Programs To Parse Programs

In this work, we study an important problem: learning programs from input-output examples. We propose a novel method to learn a neural program operating a domain-specific non-differentiable machine, and demonstrate that this method can be applied to learn programs that are significantly more complex than the ones synthesized before: programming language parsers from input-output pairs without knowing the underlying grammar. The main challenge is to train the neural program without supervision on execution traces. To tackle it, we propose: (1) LL machines and neural programs operating them to effectively regularize the space of the learned programs; and (2) a two-phase reinforcement learning-based search technique to train the model. Our evaluation demonstrates that our approach can successfully learn to parse programs in both an imperative language and a functional language, and achieve 100% test accuracy, while existing approaches' accuracies are almost 0%. This is the first successful demonstration of applying reinforcement learning to train a neural program operating a non-differentiable machine that can fully generalize to test sets on a non-trivial task.

[1]  Slav Petrov,et al.  Globally Normalized Transition-Based Neural Networks , 2016, ACL.

[2]  Razvan Pascanu,et al.  Overcoming catastrophic forgetting in neural networks , 2016, Proceedings of the National Academy of Sciences.

[3]  Alaa A. Kharbouch,et al.  Three models for the description of language , 1956, IRE Trans. Inf. Theory.

[4]  Marcin Andrychowicz,et al.  Neural Random Access Machines , 2015, ERCIM News.

[5]  Wang Ling,et al.  Learning to Compose Words into Sentences with Reinforcement Learning , 2016, ICLR.

[6]  Russell W. Quong,et al.  ANTLR: A predicated‐LL(k) parser generator , 1995, Softw. Pract. Exp..

[7]  Sumit Gulwani,et al.  Automating string processing in spreadsheets using input-output examples , 2011, POPL '11.

[8]  P. Cochat,et al.  Et al , 2008, Archives de pediatrie : organe officiel de la Societe francaise de pediatrie.

[9]  Peter Beike,et al.  The Definition Of Standard Ml Revised , 2016 .

[10]  Wojciech Zaremba,et al.  Learning Simple Algorithms from Examples , 2015, ICML.

[11]  Pushmeet Kohli,et al.  Adaptive Neural Compilation , 2016, NIPS.

[12]  Geoffrey E. Hinton,et al.  Grammar as a Foreign Language , 2014, NIPS.

[13]  Tomas Mikolov,et al.  Inferring Algorithmic Patterns with Stack-Augmented Recurrent Nets , 2015, NIPS.

[14]  Alex Graves,et al.  Neural Turing Machines , 2014, ArXiv.

[15]  Noah A. Smith,et al.  Recurrent Neural Network Grammars , 2016, NAACL.

[16]  Yoav Goldberg,et al.  Towards String-To-Tree Neural Machine Translation , 2017, ACL.

[17]  Nando de Freitas,et al.  Neural Programmer-Interpreters , 2015, ICLR.

[18]  Sumit Gulwani,et al.  Spreadsheet data manipulation using examples , 2012, CACM.

[19]  Danqi Chen,et al.  A Fast and Accurate Dependency Parser using Neural Networks , 2014, EMNLP.

[20]  Armando Solar-Lezama,et al.  Sampling for Bayesian Program Learning , 2016, NIPS.

[21]  Colin de la Higuera,et al.  Grammatical Inference: Learning Automata and Grammars , 2010 .

[22]  Kuo-Chung Tai,et al.  The Tree-to-Tree Correction Problem , 1979, JACM.

[23]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[24]  Marc Brockschmidt,et al.  Neural Program Lattices , 2016, ICLR.

[25]  Pushmeet Kohli,et al.  RobustFill: Neural Program Learning under Noisy I/O , 2017, ICML.

[26]  Lukasz Kaiser,et al.  Neural GPUs Learn Algorithms , 2015, ICLR.

[27]  Sebastian Nowozin,et al.  DeepCoder: Learning to Write Programs , 2016, ICLR.

[28]  Lihong Li,et al.  Neuro-Symbolic Program Synthesis , 2016, ICLR.

[29]  R. J. Williams,et al.  Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning , 2004, Machine Learning.

[30]  Mirella Lapata,et al.  Language to Logical Form with Neural Attention , 2016, ACL.

[31]  Pedro García,et al.  IDENTIFYING REGULAR LANGUAGES IN POLYNOMIAL TIME , 1993 .

[32]  Dawn Xiaodong Song,et al.  Making Neural Programming Architectures Generalize via Recursion , 2017, ICLR.

[33]  Navdeep Jaitly,et al.  Pointer Networks , 2015, NIPS.

[34]  Wojciech Zaremba,et al.  Reinforcement Learning Neural Turing Machines - Revised , 2015 .