Approximate Match of Rules Using Backpropagation Neural Networks

This paper presents a method for approximate match of first-order rules with unseen data. The method is useful especially in case of a multi-class problem or a noisy domain where unseen data are often not covered by the rules. Our method employs the Backpropagation Neural Network for the approximation. To build the network, we propose a technique for generating features from the rules to be used as inputs to the network. Our method has been evaluated on four domains of first-order learning problems. The experimental results show improvements of our method over the use of the original rules. We also applied our method to approximate match of propositional rules converted from an unpruned decision tree. In this case, our method can be thought of as soft-pruning of the decision tree. The results on multi-class learning domains in the UCI repository of machine learning databases show that our method performs better than standard C4.5's pruned and unpruned trees.

[1]  Raymond J. Mooney,et al.  Comparing Methods for Refining Certainty-Factor Rule-Bases , 1994, ICML.

[2]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[3]  Peter A. Flach,et al.  IBC: A First-Order Bayesian Classifier , 1999, ILP.

[4]  D. J. Newman,et al.  UCI Repository of Machine Learning Database , 1998 .

[5]  Johannes Fürnkranz,et al.  FOSSIL: A Robust Relational Learner , 1994, ECML.

[6]  Marco Botta,et al.  FONN: Combining First Order Logic with Connectionist Learning , 1997, ICML.

[7]  J. Ross Quinlan,et al.  Learning logical definitions from relations , 1990, Machine Learning.

[8]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[9]  Birgit Tausend,et al.  Representing Biases for Inductive Logic Programming , 1994, ECML.

[10]  Saso Dzeroski,et al.  Applying ILP to Diterpene Structure Elucidation from 13C NMR Spectra , 1996, Inductive Logic Programming Workshop.

[11]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[12]  Stephen Muggleton,et al.  An Experimental Comparison of Human and Machine Learning Formalisms , 1989, ML.

[13]  Marco Botta,et al.  Multiple Predicate Learning with RTL , 1995, AI*IA.

[14]  Luc De Raedt,et al.  Top-down induction of logical decision trees , 1997 .

[15]  Jorg-uwe Kietz,et al.  Controlling the Complexity of Learning in Logic through Syntactic and Task-Oriented Models , 1992 .

[16]  Jude W. Shavlik,et al.  Knowledge-Based Artificial Neural Networks , 1994, Artif. Intell..

[17]  Ivan Bratko,et al.  A knowledge base for finite element mesh design , 1994, Artif. Intell. Eng..

[18]  Francesco Bergadano,et al.  Inductive Logic Programming: From Machine Learning to Software Engineering , 1995 .

[19]  Boonserm Kijsirikul,et al.  Approximate ILP Rules by Backpropagation Neural Network: A Result on Thai Character Recognition , 1999, ILP.

[20]  Luc De Raedt,et al.  Inductive Constraint Logic , 1995, ALT.

[21]  Ashwin Srinivasan,et al.  Theories for Mutagenicity: A Study in First-Order and Feature-Based Induction , 1996, Artif. Intell..

[22]  Stephen Muggleton,et al.  Efficient Induction of Logic Programs , 1990, ALT.

[23]  Saso Dzeroski,et al.  Inductive Logic Programming: Techniques and Applications , 1993 .

[24]  Lionel Martin,et al.  Ilp-95, Leuven Mult Icn: an Empirical Multiple Predicate Learner , 1995 .

[25]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[26]  Bojan Dolsak,et al.  The Application of Inductive Logic Programming to Finite Element Mesh Design , 1992 .

[27]  Peter Clark,et al.  The CN2 Induction Algorithm , 1989, Machine Learning.

[28]  Nada Lavrac,et al.  A Study of Relevance for Learning in Deductive Databases , 1999, J. Log. Program..

[29]  Nils J. Nilsson,et al.  Principles of Artificial Intelligence , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.