Differentiable Inductive Logic Programming for Structured Examples

The differentiable implementation of logic yields a seamless combination of symbolic reasoning and deep neural networks. Recent research, which has developed a differentiable framework to learn logic programs from examples, can even acquire reasonable solutions from noisy datasets. However, this framework severely limits expressions for solutions, e.g., no function symbols are allowed, and the shapes of clauses are fixed. As a result, the framework cannot deal with structured examples. Therefore we propose a new framework to learn logic programs from noisy and structured examples, including the following contributions. First, we propose an adaptive clause search method by looking through structured space, which is defined by the generality of the clauses, to yield an efficient search space for differentiable solvers. Second, we propose for ground atoms an enumeration algorithm, which determines a necessary and sufficient set of ground atoms to perform differentiable inference functions. Finally, we propose a new method to compose logic programs softly, enabling the system to deal with complex programs consisting of several clauses. Our experiments show that our new framework can learn logic programs from noisy and structured examples, such as sequences or trees. Our framework can be scaled to deal with complex programs that consist of several clauses with function symbols.

[1]  Vítor Santos Costa,et al.  Inductive Logic Programming , 2013, Lecture Notes in Computer Science.

[2]  Claire Nédellec,et al.  Declarative Bias in ILP , 1996 .

[3]  Pedro M. Domingos,et al.  Learning the structure of Markov logic networks , 2005, ICML.

[4]  Miyuki Koshimura,et al.  Inductive Logic Programming Using a MaxSAT Solver , 2015 .

[5]  Stephen Muggleton,et al.  Meta-Interpretive Learning of Data Transformation Programs , 2015, ILP.

[6]  J. W. Lloyd,et al.  Logic for Learning , 2003, Cognitive Technologies.

[7]  Mukund Raghothaman,et al.  Synthesizing Datalog Programs Using Numerical Relaxation , 2019, IJCAI.

[8]  Luc De Raedt,et al.  Statistical Relational Artificial Intelligence: Logic, Probability, and Computation , 2016, Statistical Relational Artificial Intelligence.

[9]  R. Caferra Logic for Computer Science and Artificial Intelligence , 2011 .

[10]  Alexander G. Gray,et al.  Logical Neural Networks , 2020, ArXiv.

[11]  Shan-Hwei Nienhuys-Cheng,et al.  Foundations of Inductive Logic Programming , 1997, Lecture Notes in Computer Science.

[12]  Christopher H. Bryant,et al.  An ILP Refinement Operator for Biological Grammar Learning , 2007, ILP.

[13]  Luciano Serafini,et al.  Neural-Symbolic Computing: An Effective Methodology for Principled Integration of Machine Learning and Reasoning , 2019, FLAP.

[14]  William W. Cohen,et al.  TensorLog: A Probabilistic Database Implemented Using Deep-Learning Infrastructure , 2020, J. Artif. Intell. Res..

[15]  Fabrizio Riguzzi,et al.  Structure learning of probabilistic logic programs by searching the clause space , 2013, Theory and Practice of Logic Programming.

[16]  Akihiro Yamamoto,et al.  Learning Elementary Formal Systems , 1992, Theor. Comput. Sci..

[17]  Richard Evans,et al.  Learning Explanatory Rules from Noisy Data , 2017, J. Artif. Intell. Res..

[18]  Fabrizio Riguzzi,et al.  Lifted discriminative learning of probabilistic logic programs , 2018, Machine Learning.

[19]  Luc De Raedt Declarative Modeling for Machine Learning and Data Mining , 2012, ICFCA.

[20]  Fan Yang,et al.  Differentiable Learning of Logical Rules for Knowledge Base Reasoning , 2017, NIPS.

[21]  Marco Maggini,et al.  Relational Neural Machines , 2020, ECAI.

[22]  Sebastijan Dumancic,et al.  From Statistical Relational to Neuro-Symbolic Artificial Intelligence , 2020, IJCAI.

[23]  Luc De Raedt,et al.  DeepProbLog: Neural Probabilistic Logic Programming , 2018, BNAIC/BENELEARN.

[24]  Steven Schockaert,et al.  Lifted Relational Neural Networks: Efficient Learning of Latent Relational Structures , 2018, J. Artif. Intell. Res..

[25]  Gordon Plotkin,et al.  A Further Note on Inductive Generalization , 2008 .

[26]  Ehud Shapiro,et al.  Algorithmic Program Debugging , 1983 .

[27]  Luc De Raedt,et al.  Probabilistic Inductive Logic Programming , 2004, Probabilistic Inductive Logic Programming.

[28]  Georg Gottlob,et al.  Complexity and expressive power of logic programming , 2001, CSUR.

[29]  Akihiro Yamamoto,et al.  Using Binary Decision Diagrams to Enumerate Inductive Logic Programming Solutions , 2018, ILP Up-and-Coming / Short Papers.

[30]  Tim Rocktäschel,et al.  End-to-end Differentiable Proving , 2017, NIPS.