Fusion of Domain Knowledge with Data for Structural Learning in Object Oriented Domains

When constructing a Bayesian network, it can be advantageous to employ structural learning algorithms to combine knowledge captured in databases with prior information provided by domain experts. Unfortunately, conventional learning algorithms do not easily incorporate prior information, if this information is too vague to be encoded as properties that are local to families of variables. For instance, conventional algorithms do not exploit prior information about repetitive structures, which are often found in object oriented domains such as computer networks, large pedigrees and genetic analysis. In this paper we propose a method for doing structural learning in object oriented domains. It is demonstrated that this method is more efficient than conventional algorithms in such domains, and it is argued that the method supports a natural approach for expressing and incorporating prior information provided by domain experts.

[1]  Kevin P. Murphy,et al.  Learning the Structure of Dynamic Probabilistic Networks , 1998, UAI.

[2]  Stuart J. Russell,et al.  Adaptive Probabilistic Networks with Hidden Variables , 1997, Machine Learning.

[3]  Nir Friedman,et al.  Discovering Hidden Variables: A Structure-Based Approach , 2000, NIPS.

[4]  Ben Taskar,et al.  Probabilistic Classification and Clustering in Relational Data , 2001, IJCAI.

[5]  Kim Walden,et al.  Object-Oriented Analysis & Design , 1993, TOOLS.

[6]  Peter C. Cheeseman,et al.  Bayesian Classification (AutoClass): Theory and Results , 1996, Advances in Knowledge Discovery and Data Mining.

[7]  David Maxwell Chickering,et al.  Efficient Approximations for the Marginal Likelihood of Bayesian Networks with Hidden Variables , 1997, Machine Learning.

[8]  Nir Friedman,et al.  Being Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks , 2004, Machine Learning.

[9]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[10]  David Heckerman,et al.  A Bayesian Approach to Learning Causal Networks , 1995, UAI.

[11]  Lars Mathiassen.  Object-oriented Analysis & Design , 2000 .

[12]  Uffe Kjærulff,et al.  A Computational Scheme for Reasoning in Dynamic Probabilistic Networks , 1992, UAI.

[13]  R. Tibshirani,et al.  Monographs on statistics and applied probability , 1990 .

[14]  D. Heitjan,et al.  Distinguishing “Missing at Random” and “Missing Completely at Random” , 1996 .

[15]  Dale Schuurmans,et al.  Learning Bayesian Nets that Perform Well , 1997, UAI.

[16]  Kathryn B. Laskey,et al.  Network Engineering for Complex Belief Networks , 1996, UAI.

[17]  Kathryn B. Laskey,et al.  Network Fragments: Representing Knowledge for Constructing Probabilistic Models , 1997, UAI.

[18]  Olav Bangsø,et al.  Object Oriented Bayesian Networks: a Framework for Top-Down Specification of Large Bayesian Networks with Repetitive Structures , 2000 .

[19]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[20]  Nir Friedman,et al.  Being Bayesian about Network Structure , 2000, UAI.

[21]  R. van Engelen,et al.  Approximating Bayesian belief networks by arc removal , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Sanjoy Dasgupta,et al.  The Sample Complexity of Learning Fixed-Structure Bayesian Networks , 1997, Machine Learning.

[23]  Wray L. Buntine.  A Guide to the Literature on Learning Probabilistic Networks from Data , 1996, IEEE Trans. Knowl. Data Eng..

[24]  David J. Spiegelhalter,et al.  Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[25]  P. Green.  On Use of the EM Algorithm for Penalized Likelihood Estimation , 1990 .

[26]  David Poole,et al.  MULTIPLY SECTIONED BAYESIAN NETWORKS AND JUNCTION FORESTS FOR LARGE KNOWLEDGE‐BASED SYSTEMS , 1993, Comput. Intell..

[27]  Thomas D. Nielsen,et al.  Structural Learning in Object Oriented Domains , 2001, FLAIRS.

[28]  Avi Pfeffer,et al.  Object-Oriented Bayesian Networks , 1997, UAI.

[29]  Nir Friedman,et al.  Learning the Dimensionality of Hidden Variables , 2001, UAI.

[30]  Gregory F. Cooper,et al.  A Bayesian Method for Constructing Bayesian Belief Networks from Databases , 1991, UAI.

[31]  Nir Friedman,et al.  The Bayesian Structural EM Algorithm , 1998, UAI.

[32]  Bernard W. Silverman,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[33]  Helge Langseth,et al.  Parameter Learning in Object-Oriented Bayesian Networks , 2001, Annals of Mathematics and Artificial Intelligence.

[34]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[35]  B. Silverman.  Density estimation for statistics and data analysis , 1986 .

[36]  Daphne Koller,et al.  Probabilistic reasoning for complex systems , 1999 .

[37]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[38]  Lise Getoor,et al.  Learning Probabilistic Relational Models , 1999, IJCAI.

[39]  Finn Verner Jensen,et al.  Inference in Multiply Sectioned Bayesian Networks with Extended Shafer-Shenoy and Lazy Propagation , 1999, UAI.

[40]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[41]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[42]  F. Harary.  New directions in the theory of graphs , 1973 .

[43]  Gregory M. Provan,et al.  Knowledge Engineering for Large Belief Networks , 1994, UAI.

[44]  David Heckerman,et al.  Asymptotic Model Selection for Directed Networks with Hidden Variables , 1996, UAI.

[45]  Paul J. Krause,et al.  Learning probabilistic networks , 1999, The Knowledge Engineering Review.

[46]  Michael A. West,et al.  Bayesian Forecasting and Dynamic Models (2nd edn) , 1997, J. Oper. Res. Soc..

[47]  Nicole A. Lazar,et al.  Statistical Analysis With Missing Data , 2003, Technometrics.

[48]  Pierre-Henri Wuillemin,et al.  Top-Down Construction and Repetetive Structures Representation in Bayesian Networks , 2000, FLAIRS.

[49]  David Heckerman,et al.  Challenge: What is the Impact of Bayesian Networks on Learning? , 1997, IJCAI.

[50]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[51]  Finn Verner Jensen,et al.  Introduction to Bayesian Networks , 2008, Innovations in Bayesian Networks.

[52]  Nir Friedman,et al.  Learning Bayesian Networks with Local Structure , 1996, UAI.

[53]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[54]  Nir Friedman,et al.  On the Sample Complexity of Learning Bayesian Networks , 1996, UAI.

[55]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[56]  Yoshua Bengio,et al.  Pattern Recognition and Neural Networks , 1995 .

[57]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[58]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .