Graph Representations for Higher-Order Logic and Theorem Proving

This paper presents the first use of graph neural networks (GNNs) for higher-order proof search and demonstrates that GNNs can improve upon state-of-the-art results in this domain. Interactive, higher-order theorem provers allow for the formalization of most mathematical theories and have been shown to pose a significant challenge for deep learning. Higher-order logic is highly expressive and, even though it is well-structured with a clearly defined grammar and semantics, there still remains no well-established method to convert formulas into graph-based representations. In this paper, we consider several graphical representations of higher-order logic and evaluate them against the HOList benchmark for higher-order theorem proving.

[1]  Boris Polyak,et al.  Acceleration of stochastic approximation by averaging , 1992 .

[2]  John Harrison,et al.  HOL Light: A Tutorial Introduction , 1996, FMCAD.

[3]  Katrin Baumgartner.  Logic for Programming, Artificial Intelligence, and Reasoning , 2003, Lecture Notes in Computer Science.

[4]  Gregory N. Hullender,et al.  Learning to rank using gradient descent , 2005, ICML.

[5]  Georges Gonthier,et al.  Formal Proof—The Four- Color Theorem , 2008 .

[6]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[7]  Jeremy Avigad,et al.  A Machine-Checked Proof of the Odd Order Theorem , 2013, ITP.

[8]  Cezary Kaliszyk,et al.  Learning-Assisted Automated Reasoning with Flyspeck , 2012, Journal of Automated Reasoning.

[9]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[10]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[11]  Razvan Pascanu,et al.  A simple neural network module for relational reasoning , 2017, NIPS.

[12]  Jian Wang,et al.  Premise Selection for Theorem Proving by Deep Graph Embedding , 2017, NIPS.

[13]  Razvan Pascanu,et al.  Discovering objects and their relations from entangled scene representations , 2017, ICLR.

[14]  Cyrus Shahabi,et al.  Graph Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ArXiv.

[15]  Elad Eban,et al.  Scalable Learning of Non-Decomposable Objectives , 2016, AISTATS.

[16]  Thibault Gauthier,et al.  TacticToe: Learning to Reason with HOL4 Tactics , 2017, LPAR.

[17]  Tobias Nipkow,et al.  A FORMAL PROOF OF THE KEPLER CONJECTURE , 2015, Forum of Mathematics, Pi.

[18]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[19]  Cezary Kaliszyk,et al.  Deep Network Guided Proof Search , 2017, LPAR.

[20]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[21]  Sanjit A. Seshia,et al.  Learning Heuristics for Automated Reasoning through Deep Reinforcement Learning , 2018, ArXiv.

[22]  Cyrus Shahabi,et al.  Diffusion Convolutional Recurrent Neural Network: Data-Driven Traffic Forecasting , 2017, ICLR.

[23]  Simon Cruanes,et al.  Superposition for Lambda-Free Higher-Order Logic , 2018, IJCAR.

[24]  Richard Evans,et al.  Can Neural Networks Understand Logical Entailment? , 2018, ICLR.

[25]  Chong Wang,et al.  Neural Logic Machines , 2019, ICLR.

[26]  Jure Leskovec,et al.  How Powerful are Graph Neural Networks? , 2018, ICLR.

[27]  Dawn Xiaodong Song,et al.  GamePad: A Learning Environment for Theorem Proving , 2018, ICLR.

[28]  Nikolaj Bjørner,et al.  Guiding High-Performance SAT Solvers with Unsat-Core Predictions , 2019, SAT.

[29]  Sarah M. Loos,et al.  HOList: An Environment for Machine Learning of Higher Order Logic Theorem Proving , 2019, ICML.

[30]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Sarah M. Loos,et al.  HOList: An Environment for Machine Learning of Higher-Order Theorem Proving (extended version) , 2019, ArXiv.

[32]  Jia Deng,et al.  Learning to Prove Theorems via Interacting with Proof Assistants , 2019, ICML.

[33]  Sorin Lerner,et al.  Generating correctness proofs with neural networks , 2019, MAPL@PLDI.

[34]  Edward A. Lee,et al.  Learning Heuristics for Quantified Boolean Formulas through Reinforcement Learning , 2018, ICLR.