Premise Selection for Theorem Proving by Deep Graph Embedding

We propose a deep learning-based approach to the problem of premise selection: selecting mathematical statements relevant for proving a given conjecture. We represent a higher-order logic formula as a graph that is invariant to variable renaming but still fully preserves syntactic and semantic information. We then embed the graph into a vector via a novel embedding method that preserves the information of edge ordering. Our approach achieves state-of-the-art results on the HolStep dataset, improving the classification accuracy from 83% to 90.3%.

[1]  Alonzo Church,et al.  A formulation of the simple theory of types , 1940, Journal of Symbolic Logic.

[2]  Amy P. Felty,et al.  The Coq proof assistant user's guide : version 5.6 , 1990 .

[3]  Wolfgang Ertel,et al.  Automatic Acquisition of Search Guiding Heuristics , 1990, CADE.

[4]  Christoph Goller,et al.  Learning task-dependent distributed representations by backpropagation through structure , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[5]  Mark R. Greenstreet,et al.  Formal verification in hardware design: a survey , 1999, TODE.

[6]  C. Goller,et al.  Learning from Previous Proof Experience: A Survey , 1999 .

[7]  Stephan Schulz,et al.  Learning search control knowledge for equational deduction , 2000, DISKI.

[8]  Stephan Schulz,et al.  E - a brainiac theorem prover , 2002, AI Commun..

[9]  F. Scarselli,et al.  A new model for learning in graph domains , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[10]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[11]  Tobias Nipkow,et al.  The Isabelle Framework , 2008, TPHOLs.

[12]  Michael Norrish,et al.  seL4: formal verification of an OS kernel , 2009, SOSP '09.

[13]  Xavier Leroy,et al.  Formal verification of a realistic compiler , 2009, CACM.

[14]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[15]  Adam Naumowicz,et al.  A Brief Overview of Mizar , 2009, TPHOLs.

[16]  Andrei Voronkov,et al.  Sine Qua Non for Large Theory Reasoning , 2011, CADE.

[17]  Andrew Y. Ng,et al.  Parsing Natural Scenes and Natural Language with Recursive Neural Networks , 2011, ICML.

[18]  Jeffrey Pennington,et al.  Dynamic Pooling and Unfolding Recursive Autoencoders for Paraphrase Detection , 2011, NIPS.

[19]  John Harrison,et al.  The HOL Light Theory of Euclidean Space , 2012, Journal of Automated Reasoning.

[20]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[21]  Jeremy Avigad,et al.  A Machine-Checked Proof of the Odd Order Theorem , 2013, ITP.

[22]  Andrei Voronkov,et al.  First-Order Theorem Proving and Vampire , 2013, CAV.

[23]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[24]  Jesse Alama,et al.  Premise Selection for Mathematics by Corpus Analysis and Kernel Methods , 2011, Journal of Automated Reasoning.

[25]  James P. Bridge,et al.  Machine Learning for First-Order Theorem Proving , 2014, J. Autom. Reason..

[26]  Josef Urban,et al.  History of Interactive Theorem Proving , 2014, Computational Logic.

[27]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[28]  Mingzhe Wang,et al.  LINE: Large-scale Information Network Embedding , 2015, WWW.

[29]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[30]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[31]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[32]  Cezary Kaliszyk,et al.  FEMaLeCoP: Fairly Efficient Machine Learning Connection Prover , 2015, LPAR.

[33]  Josef Urban,et al.  MaLeS: A Framework for Automatic Tuning of Automated Theorem Provers , 2013, Journal of Automated Reasoning.

[34]  Joan Bruna,et al.  Deep Convolutional Networks on Graph-Structured Data , 2015, ArXiv.

[35]  Daniel Whalen,et al.  Holophrasm: a neural Automated Theorem Prover for higher-order logic , 2016, ArXiv.

[36]  Jure Leskovec,et al.  node2vec: Scalable Feature Learning for Networks , 2016, KDD.

[37]  Silvio Savarese,et al.  Structural-RNN: Deep Learning on Spatio-Temporal Graphs , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[39]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[40]  Chad E. Brown,et al.  Internal Guidance for Satallax , 2016, IJCAR.

[41]  Yoav Artzi,et al.  Neural Shift-Reduce CCG Semantic Parsing , 2016, EMNLP.

[42]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[43]  Josef Urban,et al.  DeepMath - Deep Sequence Models for Premise Selection , 2016, NIPS.

[44]  Josef Urban,et al.  ENIGMA: Efficient Learning-Based Inference Guiding Machine , 2017, CICM.

[45]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[46]  Tobias Nipkow,et al.  A FORMAL PROOF OF THE KEPLER CONJECTURE , 2015, Forum of Mathematics, Pi.

[47]  Cezary Kaliszyk,et al.  Deep Network Guided Proof Search , 2017, LPAR.

[48]  Cezary Kaliszyk,et al.  HolStep: A Machine Learning Dataset for Higher-order Logic Theorem Proving , 2017, ICLR.