Classifying Relational Data with Neural Networks

We introduce a novel method for relational learning with neural networks. The contributions of this paper are threefold. First, we introduce the concept of relational neural networks: feedforward networks with some recurrent components, the structure of which is determined by the relational database schema. For classifying a single tuple, they take as inputs the attribute values of not only the tuple itself, but also of sets of related tuples. We discuss several possible architectures for such networks. Second, we relate the expressiveness of these networks to the ‘aggregation vs. selection' dichotomy in current relational learners, and argue that relational neural networks can learn non-trivial combinations of aggregation and selection, a task beyond the capabilities of most current relational learners. Third, we present and motivate different possible training strategies for such networks. We present experimental results on synthetic and benchmark data sets that support our claims and yield insight in the behaviour of the proposed training strategies.

[1]  Gerson Zaverucha,et al.  Learning Logic Programs with Neural Networks , 2001, ILP.

[2]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[3]  Marco Botta,et al.  FONN: Combining First Order Logic with Connectionist Learning , 1997, ICML.

[4]  Saso Dzeroski,et al.  Diterpene Structure Elucidation from 13CNMR Spectra with Inductive Logic Programming , 1998, Appl. Artif. Intell..

[5]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[6]  P J Webros.  BACKPROPAGATION THROUGH TIME: WHAT IT DOES AND HOW TO DO IT , 1990 .

[7]  Christopher J. Merz,et al.  UCI Repository of Machine Learning Databases , 1996 .

[8]  Stefan Wrobel,et al.  Transformation-Based Learning Using Multirelational Aggregation , 2001, ILP.

[9]  Ashwin Srinivasan,et al.  An Assessment of ILP-Assisted Models for Toxicology and the PTE-3 Experiment , 1999, ILP.

[10]  Christoph Goller,et al.  Learning task-dependent distributed representations by backpropagation through structure , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[11]  Arno J. Knobbe,et al.  Involving Aggregate Functions in Multi-relational Search , 2002, PKDD.

[12]  Luc De Raedt,et al.  Attribute-Value Learning Versus Inductive Logic Programming: The Missing Links (Extended Abstract) , 1998, ILP.

[13]  Michael I. Jordan.  Attractor dynamics and parallelism in a connectionist sequential machine , 1990 .

[14]  Alessandro Sperduti,et al.  Supervised neural networks for the classification of structures , 1997, IEEE Trans. Neural Networks.

[15]  Foster J. Provost,et al.  Aggregation-based feature invention and relational concept classes , 2003, KDD '03.

[16]  Stephen Muggleton,et al.  To the international computing community: A new East-West challenge , 1994 .

[17]  Jan Ramon,et al.  Multi instance neural networks , 2000, ICML 2000.

[18]  Maurice Bruynooghe,et al.  Aggregation versus selection bias, and relational neural networks , 2003 .

[19]  Saso Dzeroski,et al.  First Order Random Forests with Complex Aggregates , 2004, ILP.

[20]  Thomas G. Dietterich,et al.  Solving the Multiple Instance Problem with Axis-Parallel Rectangles , 1997, Artif. Intell..

[21]  Alessandro Sperduti,et al.  A general framework for adaptive processing of data structures , 1998, IEEE Trans. Neural Networks.

[22]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.