Unsupervised Relational Representation Learning via Clustering: Preliminary Results

The goal of unsupervised representation learning methods is to learn a new representation of the original data, such that it makes a certain classification task easier to solve. Since their introduction in late 2000s, these methods initiated a revolution within machine learning. In this paper we present an unsupervised representation learning method for relational data. The proposed approach uses a clustering procedure to learn a new representation. Moreover, we introduce an adaptive clustering method, capable of addressing multiple interpretations of similarity in relational context. Finally, we experimentally evaluate the proposed approach. The preliminary results show the promise of the approach, as the models learned on the new representation often achieve better performance and are less complex than the ones learned on the original data representation.

[1]  Pedro M. Domingos,et al.  Learning Markov logic network structure via hypergraph lifting , 2009, ICML '09.

[2]  Lise Getoor,et al.  Collective Classification in Network Data , 2008, AI Mag..

[3]  Foster J. Provost,et al.  Aggregation-based feature invention and relational concept classes , 2003, KDD '03.

[4]  Ricardo J. G. B. Campello,et al.  Relative clustering validity criteria: A comparative overview , 2010, Stat. Anal. Data Min..

[5]  Pat Langley,et al.  Improving Efficiency by Learning Intermediate Concepts , 1989, IJCAI.

[6]  Kurt Mehlhorn,et al.  Weisfeiler-Lehman Graph Kernels , 2011, J. Mach. Learn. Res..

[7]  Lyle H. Ungar,et al.  Cluster-based concept invention for statistical relational learning , 2004, KDD.

[8]  Edwin R. Hancock,et al.  A Hypergraph Kernel from Isomorphism Tests , 2014, 2014 22nd International Conference on Pattern Recognition.

[9]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[10]  Stephen Muggleton,et al.  Meta-interpretive learning of higher-order dyadic datalog: predicate invention revisited , 2013, Machine Learning.

[11]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[12]  Hendrik Blockeel,et al.  Improving the Accuracy of Similarity Measures by Using Link Information , 2011, ISMIS.

[13]  Micah Adler,et al.  Clustering Relational Data Using Attribute and Link Information , 2003 .

[14]  Hendrik Blockeel,et al.  Top-Down Induction of First Order Logical Decision Trees , 1998, AI Commun..

[15]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[16]  Nuno A. Fonseca,et al.  Conceptual Clustering of Multi-Relational Data , 2011, ILP.

[17]  Roni Khardon,et al.  Learning from interpretations: a rooted kernel for ordered hypergraphs , 2007, ICML '07.

[18]  Pedro M. Domingos,et al.  Statistical predicate invention , 2007, ICML '07.

[19]  Pedro M. Domingos,et al.  Extracting Semantic Networks from Text Via Relational Clustering , 2008, ECML/PKDD.

[20]  Stephen Muggleton,et al.  Machine Invention of First Order Predicates by Inverting Resolution , 1988, ML.

[21]  Ashwin Srinivasan,et al.  Distinguishing Exceptions From Noise in Non-Monotonic Learning , 1992 .

[22]  Marc'Aurelio Ranzato,et al.  Sparse Feature Learning for Deep Belief Networks , 2007, NIPS.

[23]  Jun Zhou,et al.  On the Use of the Chi-Squared Distance for the Structured Learning of Graph Embeddings , 2011, 2011 International Conference on Digital Image Computing: Techniques and Applications.

[24]  Ben Taskar,et al.  Introduction to Statistical Relational Learning (Adaptive Computation and Machine Learning) , 2007 .

[25]  Mark Craven,et al.  Relational Learning with Statistical Predicate Invention: Better Models for Hypertext , 2001, Machine Learning.

[26]  William Yang Wang,et al.  A Soft Version of Predicate Invention Based on Structured Sparsity , 2015, IJCAI.

[27]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[28]  Luc De Raedt,et al.  Logical and relational learning , 2008, Cognitive Technologies.

[29]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Luc De Raedt,et al.  kLog: A Language for Logical and Relational Learning with Kernels (Extended Abstract) , 2012, IJCAI.

[31]  Dietrich Wettschereck,et al.  Relational Instance-Based Learning , 1996, ICML.

[32]  Honglak Lee,et al.  An Analysis of Single-Layer Networks in Unsupervised Feature Learning , 2011, AISTATS.

[33]  Pedro M. Domingos,et al.  Learning Markov Logic Networks Using Structural Motifs , 2010, ICML.

[34]  David Haussler,et al.  Convolution kernels on discrete structures , 1999 .

[35]  Karsten M. Borgwardt,et al.  Fast subtree kernels on graphs , 2009, NIPS.

[36]  Jesse Davis,et al.  Change of Representation for Statistical Relational Learning , 2007, IJCAI.

[37]  J. H. Ward Hierarchical Grouping to Optimize an Objective Function , 1963 .