Learning directed probabilistic logical models: ordering-search versus structure-search

We discuss how to learn non-recursive directed probabilistic logical models from relational data. This problem has been tackled before by upgrading the structure-search algorithm initially proposed for Bayesian networks. In this paper we show how to upgrade another algorithm for learning Bayesian networks, namely ordering-search. For Bayesian networks, ordering-search was found to work better than structure-search. It is non-obvious that these results carry over to the relational case, however, since there ordering-search needs to be implemented quite differently. Hence, we perform an experimental comparison of these upgraded algorithms on four relational domains. We conclude that also in the relational case ordering-search is competitive with structure-search in terms of quality of the learned models, while ordering-search is significantly faster.

[1]  Maurice Bruynooghe,et al.  Generalized Ordering-Search for Learning Directed Probabilistic Logical Models , 2006, ILP.

[2]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[3]  K. N. Dollman,et al.  - 1 , 1743 .

[4]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[5]  Daan Fierens,et al.  Learning directed probabilistic logical models from relational data , 2008, AI Commun..

[6]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine-mediated learning.

[7]  Lise Getoor,et al.  Learning Probabilistic Relational Models , 1999, IJCAI.

[8]  J. Ross Quinlan,et al.  Learning logical definitions from relations , 1990, Machine Learning.

[9]  Maurice Bruynooghe,et al.  Logical Bayesian Networks and Their Relation to Other Probabilistic Logical Models , 2005, BNAIC.

[10]  David Maxwell Chickering,et al.  Learning Equivalence Classes of Bayesian Network Structures , 1996, UAI.

[11]  Maurice Bruynooghe,et al.  Learning Directed Probabilistic Logical Models: Ordering-Search Versus Structure-Search , 2007, ECML.

[12]  J. W. Lloyd,et al.  Foundations of logic programming; (2nd extended ed.) , 1987 .

[13]  Tom M. Mitchell,et al.  Learning to Extract Symbolic Knowledge from the World Wide Web , 1998, AAAI/IAAI.

[14]  Craig Boutilier,et al.  Context-Specific Independence in Bayesian Networks , 1996, UAI.

[15]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[16]  Luc De Raedt,et al.  Probabilistic Inductive Logic Programming , 2004, Probabilistic Inductive Logic Programming.

[17]  Saso Dzeroski,et al.  First order random forests: Learning relational classifiers with complex aggregates , 2006, Machine Learning.

[18]  Luc De Raedt,et al.  Basic Principles of Learning Bayesian Logic Programs , 2008, Probabilistic Inductive Logic Programming.

[19]  John Wylie Lloyd,et al.  Foundations of Logic Programming , 1987, Symbolic Computation.

[20]  Weng-Keen Wong,et al.  Structure Refinement in First Order Conditional Influence Language , 2006 .

[21]  Luc De Raedt,et al.  Towards Combining Inductive Logic Programming with Bayesian Networks , 2001, ILP.

[22]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[23]  Daphne Koller,et al.  Ordering-Based Search: A Simple and Effective Algorithm for Learning Bayesian Networks , 2005, UAI.

[24]  Luc De Raedt,et al.  Probabilistic Inductive Logic Programming , 2004, ALT.

[25]  Manfred Jaeger,et al.  Parameter learning for relational Bayesian networks , 2007, ICML '07.

[26]  Raymond J. Mooney,et al.  Mapping and Revising Markov Logic Networks for Transfer Learning , 2007, AAAI.

[27]  Maurice Bruynooghe,et al.  A Comparison of Pruning Criteria for Probability Trees , 2007 .

[28]  Luc De Raedt,et al.  Lookahead and Discretization in ILP , 1997, ILP.

[29]  Matthew Richardson,et al.  Markov logic networks , 2006, Machine Learning.

[30]  Maurice Bruynooghe,et al.  Generalized ordering-search for learning directed probabilistic logical models , 2007, Machine Learning.