论文信息 - On Local Optima in Learning Bayesian Networks

On Local Optima in Learning Bayesian Networks

This paper proposes and evaluates the k-greedy equivalence search algorithm (KES) for learning Bayesian networks (BNs) from complete data. The main characteristic of KES is that it allows a trade-off between greediness and randomness, thus exploring different good local optima when run repeatedly. When greediness is set at maximum, KES corresponds to the greedy equivalence search algorithm (GES). When greediness is kept at minimum, we prove that under mild conditions KES asymptotically returns any inclusion optimal BN with nonzero probability. Experimental results for both synthetic and real data are reported showing that KES finds a better local optimum than GES considerably often. Additionally, these results illustrate that the number of different local optima is usually huge.

José M. Peña | Tomas Kocka | Jens Dalgaard Nielsen

[1] David J. Spiegelhalter,et al. Probabilistic Networks and Expert Systems , 1999, Information Science and Statistics.

[2] Michael I. Jordan. Graphical Models , 2003 .

[3] Robert Castelo,et al. On Inclusion-Driven Learning of Bayesian Networks , 2003, J. Mach. Learn. Res..

[4] Martin Beibel. Selection of Informative Genes in Gene Expression Based Diagnosis: A Nonparametric Approach , 2000, ISMDA.

[5] P. Spirtes,et al. Causation, prediction, and search , 1993 .

[6] Nir Friedman,et al. Class discovery in gene expression data , 2001, RECOMB.

[7] David Maxwell Chickering,et al. Optimal Structure Identification With Greedy Search , 2002, J. Mach. Learn. Res..

[8] David Maxwell Chickering,et al. Finding Optimal Bayesian Networks , 2002, UAI.

[9] Edward H. Herskovits,et al. Computer-based probabilistic-network construction , 1992 .

[10] Tom Burr,et al. Causation, Prediction, and Search , 2003, Technometrics.

[11] Gregory F. Cooper,et al. A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[12] J. Mesirov,et al. Molecular classification of cancer: class discovery and class prediction by gene expression monitoring. , 1999, Science.

[13] David Maxwell Chickering,et al. A Transformational Characterization of Equivalent Bayesian Network Structures , 1995, UAI.

[14] C. Meek,et al. Graphical models: selecting causal and statistical models , 1997 .