Predicate Invention by Learning From Failures

Discovering novel high-level concepts is one of the most important steps needed for human-level AI. In inductive logic programming (ILP), discovering novel-high level concepts is known as predicate invention (PI). Although seen as crucial since the founding of ILP, PI is notoriously difficult and most ILP systems do not support it. In this paper, we introduce POPPI, an ILP system that formulates the PI problem as an answer set programming problem. Our experiments show that (i) PI can drastically improve learning performance when useful, (ii) PI is not too costly when unnecessary, and (iii) POPPI can substantially outperform existing ILP systems.

[1]  F. Luo.  Cognitive Technologies , 2021 .

[2]  Stephen Muggleton,et al.  Bias reformulation for one-shot function induction , 2014, ECAI.

[3]  Peter Schüller,et al.  Best-effort inductive logic programming via fine-grained cost-based hypothesis generation , 2017, Machine Learning.

[4]  autoepistemic Zogic.  Logic programming and negation : a survey , 2001 .

[5]  Krysia Broda,et al.  Inductive Learning of Answer Set Programs , 2014, JELIA.

[6]  Andrew Cropper,et al.  Knowledge Refactoring for Inductive Program Synthesis , 2020 .

[7]  Martin Gebser,et al.  Multi-shot ASP solving with clingo , 2017, Theory and Practice of Logic Programming.

[8]  Stephen Muggleton,et al.  Complete Bottom-Up Predicate Invention in Meta-Interpretive Learning , 2020, IJCAI.

[9]  Stefan Kramer,et al.  A Brief History of Learning Symbolic Higher-Level Representations from Data (And a Curious Look Forward) , 2020, IJCAI.

[10]  Andrew Cropper,et al.  Logical reduction of metarules , 2019, Machine Learning.

[11]  Stephen Muggleton,et al.  Machine Invention of First Order Predicates by Inverting Resolution , 1988, ML.

[12]  William Yang Wang,et al.  A Soft Version of Predicate Invention Based on Structured Sparsity , 2015, IJCAI.

[13]  Katsumi Inoue,et al.  Meta-Interpretive Learning Using HEX-Programs , 2019, IJCAI.

[14]  Wannes Meert,et al.  Learning Relational Representations with Auto-encoding Logic Programs , 2019, IJCAI.

[15]  José Hernández-Orallo,et al.  Making sense of sensory input , 2019, Artif. Intell..

[16]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[17]  Joshua S. Rule,et al.  The Child as Hacker , 2020, Trends in Cognitive Sciences.

[18]  Hendrik Blockeel,et al.  Clustering-Based Relational Unsupervised Representation Learning with an Explicit Distributed Representation , 2016, IJCAI.

[19]  Vítor Santos Costa,et al.  Inductive Logic Programming , 2013, Lecture Notes in Computer Science.

[20]  Stephen Muggleton,et al.  Learning Higher-Order Logic Programs through Abstraction and Invention , 2016, IJCAI.

[21]  Andrew Cropper,et al.  Playgol: learning programs through play , 2019, IJCAI.

[22]  Stephen Muggleton,et al.  Meta-interpretive learning of higher-order dyadic datalog: predicate invention revisited , 2013, Machine Learning.

[23]  Richard Evans,et al.  Learning Explanatory Rules from Noisy Data , 2017, J. Artif. Intell. Res..

[24]  J. Lloyd.  Foundations of Logic Programming , 1984, Symbolic Computation.

[25]  Pedro M. Domingos,et al.  Statistical predicate invention , 2007, ICML '07.

[26]  Rolf Morel,et al.  Learning programs by learning from failures , 2020, Mach. Learn..

[27]  G. Plotkin.  Automatic Methods of Inductive Inference , 1972 .

[28]  Martin Gebser,et al.  Answer Set Solving in Practice , 2012, Answer Set Solving in Practice.

[29]  Andrew Cropper,et al.  Turning 30: New Ideas in Inductive Logic Programming , 2020, ArXiv.

[30]  Irene Stahl,et al.  The appropriateness of predicate invention as bias shift operation in ILP , 1995, Machine Learning.

[31]  Katsumi Inoue,et al.  ILP turns 20 - Biography and future challenges , 2012, Mach. Learn..

[32]  Stuart Russell.  Human Compatible: Artificial Intelligence and the Problem of Control , 2019 .