Learning Bayesian Networks by Floating Search Methods

In this work, two novel sequential algorithms for learning Bayesian networks are proposed. The presented sequential search methods are an adaptation of a pair of algorithms proposed to feature subset selection: Sequential Forward Floating Selection and Sequential Backward Floating Selection. As far as we know, these algorithms have never been used for learning Bayesian networks. An empirical comparison among the results of the proposed algorithms and the results of two sequential algorithm (the classical B-algorithm and its extension, the B3 algorithm) is carried out over four databases from literature. The results show promising results for the floating approach to the learning Bayesian network problem.

[1]  Finn V. Jensen,et al.  Bayesian Networks and Decision Graphs , 2001, Statistics for Engineering and Information Science.

[2]  Enrique F. Castillo,et al.  Expert Systems and Probabilistic Network Models , 1996, Monographs in Computer Science.

[3]  Pedro Larrañaga,et al.  Structure Learning of Bayesian Networks by Genetic Algorithms: A Performance Analysis of Control Parameters , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[4]  Jose Miguel Puerta,et al.  Ant colony optimization for learning Bayesian networks , 2002, Int. J. Approx. Reason..

[5]  Pedro Larrañaga,et al.  Learning Bayesian network structures by searching for the best ordering with genetic algorithms , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[6]  Robert Castelo,et al.  Improved learning of Bayesian networks , 2001, UAI.

[7]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine Learning.

[8]  Wray L. Buntine.  Theory Refinement on Bayesian Networks , 1991, UAI.

[9]  Wray L. Buntine.  A Guide to the Literature on Learning Probabilistic Networks from Data , 1996, IEEE Trans. Knowl. Data Eng..

[10]  David A. Bell,et al.  Learning Bayesian networks from data: An information-theory based approach , 2002, Artif. Intell..

[11]  H. B. Mann,et al.  On a Test of Whether one of Two Random Variables is Stochastically Larger than the Other , 1947 .

[12]  A. H. Murphy,et al.  Hailfinder: A Bayesian system for forecasting severe weather , 1996 .

[13]  Robert G. Cowell,et al.  Conditions Under Which Conditional Independence and Scoring Methods Lead to Identical Selection of Bayesian Network Models , 2001, UAI.

[14]  Carlos Cotta,et al.  Towards a More Efficient Evolutionary Induction of Bayesian Networks , 2002, PPSN.

[15]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[16]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[17]  Grace Wahba.  [Monotone Regression Splines in Action]: Comment , 1988 .

[18]  Luis M. de Campos,et al.  A new approach for learning belief networks using independence criteria , 2000, Int. J. Approx. Reason..

[19]  Jin Tian,et al.  A Branch-and-Bound Algorithm for MDL Learning Bayesian Networks , 2000, UAI.

[20]  Josef Kittler,et al.  Floating search methods in feature selection , 1994, Pattern Recognit. Lett..

[21]  Nir Friedman,et al.  On the application of the bootstrap for computing confidence measures on features of induced Bayesian networks , 1999, AISTATS.

[22]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[23]  Pedro Larrañaga,et al.  Unsupervised Learning Of Bayesian Networks Via Estimation Of Distribution Algorithms: An Application To Gene Expression Data Clustering , 2004, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[24]  Nir Friedman,et al.  Being Bayesian About Network Structure. A Bayesian Approach to Structure Discovery in Bayesian Networks , 2004, Machine Learning.

[25]  David Maxwell Chickering,et al.  Learning Equivalence Classes of Bayesian Network Structures , 1996, UAI.

[26]  P. Spirtes,et al.  Causation, prediction, and search , 1993 .

[27]  Harald Steck.  On the Use of Skeletons when Learning in Bayesian Networks , 2000, UAI.

[28]  David Heckerman,et al.  Learning Bayesian Networks: Search Methods and Experimental Results , 1995 .

[29]  Christian Borgelt,et al.  An Empirical Investigation of the K2 Metric , 2001, ECSQARU.

[30]  Stuart J. Russell,et al.  Adaptive Probabilistic Networks with Hidden Variables , 1997, Machine Learning.

[31]  Luis M. de Campos,et al.  The Search of Causal Orderings: A Short Cut for Learning Belief Networks , 2001, ECSQARU.

[32]  A. Dawid.  Conditional Independence in Statistical Theory , 1979 .

[33]  Allan Leck Jensen,et al.  MIDAS: An Influence Diagram for Management of Mildew in Winter Wheat , 1996, UAI.

[34]  Michael D. Perlman,et al.  Enumerating Markov Equivalence Classes of Acyclic Digraph Models , 2001, UAI.

[35]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[36]  J. Kittler,et al.  Feature Set Search Alborithms , 1978 .

[37]  Jose Miguel Puerta,et al.  Stochastic Local Algorithms for Learning Belief Networks: Searching in the Space of the Orderings , 2001, ECSQARU.

[38]  Jérôme Habrant,et al.  Structure Learning of Bayesian Networks from Databases by Genetic Algorithms-Application to Time Series Prediction in Finance , 1999, ICEIS.

[39]  Pedro Larrañaga,et al.  Learning Bayesian networks in the space of structures by estimation of distribution algorithms , 2003, Int. J. Intell. Syst..

[40]  Gregory F. Cooper,et al.  An Entropy-driven System for Construction of Probabilistic Expert Systems from Databases , 1990, UAI.