Ensemble parameter estimation for graphical models

Parameter Estimation is one of the key issues involved in the discovery of graphical models from data. Current state of the art methods have demonstrated their abilities in different kind of graphical models. In this paper, we introduce ensemble learning into the process of parameter estimation, and examine ensemble parameter estimation methods for different kind of graphical models under complete data set and incomplete data set. We provide experimental results which show that ensemble method can achieve an improved result over the base parameter estimation method in terms of accuracy. In addition, the method is amenable to parallel or distributed processing, which is an important characteristic for data mining in large data sets.

[1]  R. P. McDonald,et al.  Structural Equations with Latent Variables , 1989 .

[2]  David J. Spiegelhalter,et al.  Sequential updating of conditional probabilities on directed graphical structures , 1990, Networks.

[3]  Michael I. Jordan.  Learning in Graphical Models , 1999, NATO ASI Series.

[4]  Richard Maclin,et al.  Ensembles as a Sequence of Classifiers , 1997, IJCAI.

[5]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[6]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[7]  S. Lauritzen.  The EM algorithm for graphical association models with missing data , 1995 .

[8]  Harris Drucker,et al.  Improving Performance in Neural Networks Using a Boosting Algorithm , 1992, NIPS.

[9]  Pádraig Cunningham,et al.  Stability problems with artificial neural networks and the ensemble solution , 2000, Artif. Intell. Medicine.

[10]  Leo Breiman,et al.  Bagging Predictors , 1996, Machine Learning.

[11]  Joseph L Schafer,et al.  Analysis of Incomplete Multivariate Data , 1997 .

[12]  Kevin B. Korb,et al.  Causal Discovery via MML , 1996, ICML.

[13]  J. Ross Quinlan,et al.  Bagging, Boosting, and C4.5 , 1996, AAAI/IAAI, Vol. 1.

[14]  Nathan Intrator,et al.  Classification of seismic signals by integrating ensembles of neural networks , 1998, IEEE Trans. Signal Process..

[15]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[16]  Gang Li,et al.  Linear Causal Model discovery using the MML criterion , 2002, 2002 IEEE International Conference on Data Mining, 2002. Proceedings..

[17]  Jianchang Mao,et al.  A case study on bagging, boosting and basic ensembles of neural networks for OCR , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[18]  Kevin Murphy,et al.  Bayes net toolbox for Matlab , 1999 .

[19]  D. Rubin,et al.  Multiple Imputation for Nonresponse in Surveys , 1989 .

[20]  Nir Friedman,et al.  Being Bayesian about Network Structure , 2000, UAI.