Predicate Invention and Learning from Positive Examples Only

Previous bias shift approaches to predicate invention are not applicable to learning from positive examples only, if a complete hypothesis can be found in the given language, as negative examples are required to determine whether new predicates should be invented or not. One approach to this problem is presented, MERLIN 2.0, which is a successor of a system in which predicate invention is guided by sequences of input clauses in SLD-refutations of positive and negative examples w.r.t. an overly general theory. In contrast to its predecessor which searches for the minimal finite-state automaton that can generate all positive and no negative sequences, MERLIN 2.0 uses a technique for inducing Hidden Markov Models from positive sequences only. This enables the system to invent new predicates without being triggered by negative examples. Another advantage of using this induction technique is that it allows for incremental learning. Experimental results are presented comparing MERLIN 2.0 with the positive only learning framework of Progol 4.2 and comparing the original induction technique with a new version that produces deterministic Hidden Markov Models. The results show that predicate invention may indeed be both necessary and possible when learning from positive examples only as well as it can be beneficial to keep the induced model deterministic.

[1]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[2]  Jerome A. Feldman,et al.  On the Synthesis of Finite-State Machines from Samples of Their Behavior , 1972, IEEE Transactions on Computers.

[3]  J. Lloyd.  Foundations of Logic Programming , 1984, Symbolic Computation.

[4]  J. W. Lloyd,et al.  Foundations of logic programming; (2nd extended ed.) , 1987 .

[5]  John Wylie Lloyd,et al.  Foundations of Logic Programming , 1987, Symbolic Computation.

[6]  Rüdiger Wirth,et al.  Constraints on Predicate Invention , 1991, ML.

[7]  Stephen Muggleton,et al.  Non-monotonic learning , 1991 .

[8]  Masayuki Numao,et al.  Discrimination-Based Constructive Induction of Logic Programs , 1992, AAAI.

[9]  Stan Matwin,et al.  Constructive Inductive Logic Programming , 1993, IJCAI.

[10]  Andreas Stolcke,et al.  Best-first Model Merging for Hidden Markov Model Induction , 1994, ArXiv.

[11]  S. Muggleton.  Stochastic Logic Programs , 1996 .

[12]  Henrik Boström,et al.  Theory-Guideed Induction of Logic Programs by Inference of Regular Languages , 1996, ICML.

[13]  Stephen Muggleton,et al.  Learning from Positive Data , 1996, Inductive Logic Programming Workshop.

[14]  Stephen Muggleton.  Inductive Logic Programming: 6th International Workshop, ILP-96, Stockholm, Sweden, August 26-28, 1996, Selected Papers , 1997 .

[15]  Christos H. Papadimitriou,et al.  Elements of the Theory of Computation , 1997, SIGA.