Adapting Bayes Network Structures to Non-stationary Domains

When an incremental structural learning method gradually modifies a Bayesian network (BN) structure to fit a sequential stream of observations, we call the process structural adaptation. Structural adaptation is useful when the learner is set to work in an unknown environment, where a BN is gradually being constructed as observations of the environment are made. Existing algorithms for incremental learning assume that the samples in the database have been drawn from a single underlying distribution. In this paper we relax this assumption, so that the underlying distribution can change during the sampling of the database. The proposed method can thus be used in unknown environments, where it is not even known whether the dynamics of the environment are stable. We state formal correctness results for our method, and demonstrate its feasibility experimentally.

[1]  Nir Friedman,et al.  Bayesian Network Classifiers , 1997, Machine Learning.

[2]  João Gama,et al.  An Adaptive Prequential Learning Framework for Bayesian Network Classifiers , 2006, PKDD.

[3]  Fabio Gagliardi Cozman,et al.  Random Generation of Bayesian Networks , 2002, SBIA.

[4]  Sebastian Thrun,et al.  Bayesian Network Induction via Local Neighborhoods , 1999, NIPS.

[5]  Thomas D. Nielsen,et al.  Alert Systems for Production Plants: A Methodology Based on Conflict Analysis , 2005, ECSQARU.

[6]  Jesper Tegnér,et al.  Scalable, Efficient and Correct Learning of Markov Boundaries Under the Faithfulness Assumption , 2005, ECSQARU.

[7]  Steffen L. Lauritzen,et al.  aHUGIN: A System Creating Adaptive Causal Probabilistic Networks , 1992, UAI.

[8]  David Heckerman,et al.  A Tutorial on Learning with Bayesian Networks , 1999, Innovations in Bayesian Networks.

[9]  Frank Jensen,et al.  Analysis in HUGIN of data conflict , 1990, UAI.

[10]  P. Spirtes,et al.  Causation, Prediction, and Search, 2nd Edition , 2001 .

[11]  Wai Lam,et al.  Bayesian Network Refinement Via Machine Learning Approach , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Jayme Luiz Szwarcfiter,et al.  A search strategy for the elementary cycles of a directed graph , 1976 .

[13]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[14]  W. Press,et al.  Numerical Recipes in C++: The Art of Scientific Computing (2nd edn)1 Numerical Recipes Example Book (C++) (2nd edn)2 Numerical Recipes Multi-Language Code CD ROM with LINUX or UNIX Single-Screen License Revised Version3 , 2003 .

[15]  Constantin F. Aliferis,et al.  Identifying Markov blankets with decision tree induction , 2003, Third IEEE International Conference on Data Mining.

[16]  Bruno Caprile.  Uniformly Generating Distribution Functions for Discrete Random Variables , 2000 .

[17]  Wray L. Buntine.  Theory Refinement on Bayesian Networks , 1991, UAI.

[18]  Nir Friedman,et al.  Sequential Update of Bayesian Network Structure , 1997, UAI.

[19]  Fabio Gagliardi Cozman,et al.  Generating Random Bayesian Networks with Constraints on Induced Width , 2004, ECAI.

[20]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[21]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[22]  Tom Burr,et al.  Causation, Prediction, and Search , 2003, Technometrics.

[23]  Serafín Moral,et al.  Qualitative combination of Bayesian networks , 2003, Int. J. Intell. Syst..

[24]  Søren Holbech Nielsen,et al.  Adapting Bayes Nets to Non-stationary Probability Distributions , 2006 .

[25]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems , 1988 .

[26]  Judea Pearl,et al.  Equivalence and Synthesis of Causal Models , 1990, UAI.

[27]  J. Alcobé.  Incremental Hill-Climbing Search Applied to Bayesian Network Structure Learning , 2004 .

[28]  Arie M. C. A. Koster,et al.  Treewidth: Computational Experiments , 2001, Electron. Notes Discret. Math..

[29]  Steffen L. Lauritzen,et al.  Independence properties of directed markov fields , 1990, Networks.

[30]  Wai Lam,et al.  Using New Data to Refine a Bayesian Network , 1994, UAI.