Structure Refinement in First Order Conditional Influence Language

In this paper, we present preliminary results from learning the structure of first-order conditional influence statements from data for the purpose of classification. In order to reduce the search space over structures, we formulate and address the structure learning problem as a problem of refining the structure of a first-order probabilistic program using training data. We use variants of the conditional BIC scoring metric to refine the program to best fit the data. We use a previously introduced language called FOCIL which consists of statements that can be instantiated and composed into a propositional Bayesian network. The results on a synthetic dataset and a real-world task show that the algorithm achieves error rates comparable to the gold standard program with a reasonable amount of training data.

[1]  Manfred Jaeger,et al.  Relational Bayesian Networks , 1997, UAI.

[2]  Luc De Raedt,et al.  Bayesian Logic Programs , 2001, ILP Work-in-progress reports.

[3]  Thomas G. Dietterich,et al.  TaskTracer: a desktop environment to support multi-tasking knowledge workers , 2005, IUI.

[4]  F. Harary.  New directions in the theory of graphs , 1973 .

[5]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[6]  David Maxwell Chickering,et al.  Learning Equivalence Classes of Bayesian Network Structures , 1996, UAI.

[7]  Jennifer Neville,et al.  Dependency networks for relational data , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[8]  Adrian E. Raftery,et al.  Approximate Bayes Factors for Image Segmentation: The Pseudolikelihood Information Criterion (PLIC) , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Lise Getoor,et al.  Learning Probabilistic Relational Models , 1999, IJCAI.

[10]  Guillaume Bouchard,et al.  Selection of generative models in classification , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Jennifer Neville,et al.  Learning relational probability trees , 2003, KDD '03.

[12]  Eric E. Altendorf,et al.  First order conditional influence language , 2005 .

[13]  Russell Greiner,et al.  Discriminative Model Selection for Belief Net Structures , 2005, AAAI.

[14]  David Maxwell Chickering,et al.  Dependency Networks for Inference, Collaborative Filtering, and Data Visualization , 2000, J. Mach. Learn. Res..

[15]  Pedro M. Domingos,et al.  Learning the structure of Markov logic networks , 2005, ICML.

[16]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[17]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[18]  A. Raftery.  Bayesian Model Selection in Social Research , 1995 .

[19]  Ben Taskar,et al.  Markov Logic: A Unifying Framework for Statistical Relational Learning , 2007 .

[20]  Thomas G. Dietterich,et al.  Learning first-order probabilistic models with combining rules , 2005, Annals of Mathematics and Artificial Intelligence.

[21]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[22]  Pedro M. Domingos,et al.  Learning Bayesian network classifiers by maximizing conditional likelihood , 2004, ICML.