Bridging Weighted Rules and Graph Random Walks for Statistical Relational Models

The aim of statistical relational learning is to learn statistical models from relational or graph-structured data. Three main statistical relational learning paradigms include weighted rule learning, random walks on graphs, and tensor factorization. These paradigms have been mostly developed and studied in isolation for many years, with few works attempting at understanding the relationship among them or combining them. In this article, we study the relationship between the path ranking algorithm (PRA), one of the most well-known relational learning methods in the graph random walk paradigm, and relational logistic regression (RLR), one of the recent developments in weighted rule learning. We provide a simple way to normalize relations and prove that relational logistic regression using normalized relations generalizes the path ranking algorithm. This result provides a better understanding of relational learning, especially for the weighted rule learning and graph random walk paradigms. It opens up the possibility of using the more flexible RLR rules within PRA models and even generalizing both by including normalized and unnormalized relations in the same model.

[1]  Stephen E. Robertson,et al.  Understanding inverse document frequency: on theoretical arguments for IDF , 2004, J. Documentation.

[2]  Michael McCool,et al.  A Short Introduction , 2018, Passionate Embrace.

[3]  Lise Getoor,et al.  A short introduction to probabilistic soft logic , 2012, NIPS 2012.

[4]  Oren Etzioni,et al.  Markov Logic Networks for Natural Language Question Answering , 2015, ArXiv.

[5]  Lise Getoor,et al.  Lifted graphical models: a survey , 2011, Machine Learning.

[6]  Hans-Peter Kriegel,et al.  Factorizing YAGO: scalable machine learning for linked data , 2012, WWW.

[7]  Ondrej Kuzelka,et al.  Lifted Relational Neural Networks , 2015, CoCo@NIPS.

[8]  Dániel Fogaras,et al.  Towards Scaling Fully Personalized PageRank , 2004, WAW.

[9]  Dejing Dou,et al.  Weakly Supervised Tweet Stance Classification by Relational Bootstrapping , 2016, EMNLP.

[10]  Gerard Salton,et al.  Term-Weighting Approaches in Automatic Text Retrieval , 1988, Inf. Process. Manag..

[11]  Andrew McCallum,et al.  Introduction to Statistical Relational Learning , 2007 .

[12]  Jason Weston,et al.  Translating Embeddings for Modeling Multi-relational Data , 2013, NIPS.

[13]  Dániel Fogaras,et al.  Towards Scaling Fully Personalized PageRank: Algorithms, Lower Bounds, and Experiments , 2005, Internet Math..

[14]  Tom M. Mitchell,et al.  Random Walk Inference and Learning in A Large Scale Knowledge Base , 2011, EMNLP.

[15]  Yizhou Sun,et al.  Personalized entity recommendation: a heterogeneous information network approach , 2014, WSDM.

[16]  Alexander J. Smola,et al.  Go for a Walk and Arrive at the Answer: Reasoning Over Paths in Knowledge Bases using Reinforcement Learning , 2017, ICLR.

[17]  Estevam R. Hruschka,et al.  Toward an Architecture for Never-Ending Language Learning , 2010, AAAI.

[18]  Ni Lao,et al.  Fast query execution for retrieval models based on path-constrained random walks , 2010, KDD.

[19]  Ni Lao,et al.  Relational retrieval using a combination of path-constrained random walks , 2010, Machine Learning.

[20]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[21]  Seyed Mehran Kazemi,et al.  A Learning Algorithm for Relational Logistic Regression: Preliminary Results , 2016, ArXiv.

[22]  Seyed Mehran Kazemi,et al.  RelNN: A Deep Neural Model for Relational Learning , 2017, AAAI.

[23]  Matthew Richardson,et al.  Markov Logic , 2008, Probabilistic Inductive Logic Programming.

[24]  Peter J. F. Lucas,et al.  Generalising the Interaction Rules in Probabilistic Logic , 2011, IJCAI.

[25]  Evgeniy Gabrilovich,et al.  A Review of Relational Machine Learning for Knowledge Graphs , 2015, Proceedings of the IEEE.

[26]  Seyed Mehran Kazemi,et al.  Finding a Record in a Database , 2017 .

[27]  James R. Foulds,et al.  Joint Models of Disagreement and Stance in Online Debate , 2015, ACL.

[28]  J. Ross Quinlan,et al.  Learning logical definitions from relations , 1990, Machine Learning.

[29]  Luc De Raedt,et al.  Solving Probability Problems in Natural Language , 2017, IJCAI.

[30]  Pedro M. Domingos,et al.  Entity Resolution with Markov Logic , 2006, Sixth International Conference on Data Mining (ICDM'06).

[31]  Luc De Raedt,et al.  ProbLog: A Probabilistic Prolog and its Application in Link Discovery , 2007, IJCAI.

[32]  Xueyan Jiang,et al.  Reducing the Rank in Relational Factorization Models by Including Observable Patterns , 2014, NIPS.

[33]  Lise Getoor,et al.  Generic Statistical Relational Entity Resolution in Knowledge Graphs , 2016, ArXiv.

[34]  Guillaume Bouchard,et al.  Complex Embeddings for Simple Link Prediction , 2016, ICML.

[35]  Huanbo Luan,et al.  Modeling Relation Paths for Representation Learning of Knowledge Bases , 2015, EMNLP.

[36]  Kristian Kersting,et al.  Relational Logistic Regression: The Directed Analog of Markov Logic Networks , 2014, StarAI@AAAI.

[37]  Lise Getoor,et al.  Knowledge Graph Identification , 2013, SEMWEB.

[38]  Luc De Raedt,et al.  Statistical Relational Artificial Intelligence: Logic, Probability, and Computation , 2016, Statistical Relational Artificial Intelligence.

[39]  Andrew McCallum,et al.  Relation Extraction with Matrix Factorization and Universal Schemas , 2013, NAACL.

[40]  Seyed Mehran Kazemi,et al.  Comparing Aggregators for Relational Probabilistic Models , 2017, ArXiv.

[41]  Patrick Pantel,et al.  FactRank: Random Walks on a Web of Facts , 2010, COLING.

[42]  Lise Getoor,et al.  Collective entity resolution in relational data , 2007, TKDD.