Neural Relational Learning Through Semi-Propositionalization of Bottom Clauses

Relational learning can be described as the task of learning first-order logic rules from examples. It has enabled a number of new machine learning applications, e.g. graph mining and link analysis in social networks. The CILP++ system is a neural-symbolic system which can perform efficient relational learning, by being able to process first-order logic knowledge into a neural network. CILP++ relies on BCP, a recently discovered propositionalization algorithm, to perform relational learning. However, efficient knowledge extraction from such networks is an open issue and features generated by BCP do not have an independent relational description, which prevents sound knowledge extraction from such networks. We present a methodology for generating independent propositional features for BCP by using semi-propositionalization of bottom clauses. Empirical results obtained in comparison with the original version of BCP show that this approach has comparable accuracy and runtimes, while allowing proper relational knowledge representation of features for knowledge extraction from CILP++ networks.

[1]  Luc De Raedt,et al.  Inductive Logic Programming: Theory and Methods , 1994, J. Log. Program..

[2]  Ashwin Srinivasan,et al.  Relating chemical activity to structure: An examination of ILP successes , 1995, New Generation Computing.

[3]  Jude W. Shavlik,et al.  in Advances in Neural Information Processing , 1996 .

[4]  S. Džeroski,et al.  Relational Data Mining , 2001, Springer Berlin Heidelberg.

[5]  Artur S. d'Avila Garcez,et al.  The Connectionist Inductive Learning and Logic Programming System , 1999, Applied Intelligence.

[6]  Peter A. Flach,et al.  Comparative Evaluation of Approaches to Propositionalization , 2003, ILP.

[7]  Dov M. Gabbay,et al.  Neural-Symbolic Cognitive Reasoning , 2008, Cognitive Technologies.

[8]  Artur S. d'Avila Garcez,et al.  Fast relational learning using bottom clause propositionalization with artificial neural networks , 2013, Machine Learning.

[9]  J. Parke,et al.  A Descriptive Analysis of Demographic and Behavioral Data from Internet Gamblers and Those Who Self-exclude from Online Gambling Platforms , 2015, Journal of Gambling Studies.

[10]  Stephen Muggleton,et al.  The lattice structure and refinement operators for the hypothesis space bounded by a bottom clause , 2009, Machine Learning.

[11]  Peter A. Flach,et al.  An extended transformation approach to inductive logic programming , 2001, ACM Trans. Comput. Log..

[12]  Nada Lavrac,et al.  Propositionalization-based relational subgroup discovery with RSD , 2006, Machine Learning.

[13]  Bernard Widrow,et al.  The basic ideas in neural networks , 1994, CACM.

[14]  Luc De Raedt,et al.  Logical and relational learning , 2008, Cognitive Technologies.

[15]  Shan-Hwei Nienhuys-Cheng,et al.  Foundations of Inductive Logic Programming , 1997, Lecture Notes in Computer Science.

[16]  Stephen Muggleton,et al.  QG/GA: A Stochastic Search for Progol , 2006, ILP.

[17]  Gerson Zaverucha,et al.  Learning Theories Using Estimation Distribution Algorithms and (Reduced) Bottom Clauses , 2011, ILP.

[18]  Stephen Muggleton,et al.  Inverse entailment and progol , 1995, New Generation Computing.

[19]  Krysia Broda,et al.  Neural-symbolic learning systems - foundations and applications , 2012, Perspectives in neural computing.

[20]  Jude W. Shavlik,et al.  Learning an Approximation to Inductive Logic Programming Clause Evaluation , 2004, ILP.

[21]  Saso Dzeroski,et al.  Inductive Logic Programming: Techniques and Applications , 1993 .

[22]  Ashwin Srinivasan,et al.  Mutagenesis: ILP experiments in a non-determinate biological domain , 1994 .

[23]  Stephen Muggleton,et al.  Learning optimal chess strategies , 1994, Machine Intelligence 13.

[24]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .