A Branch-and-Bound Algorithm for MDL Learning Bayesian Networks

This paper extends the work in [Suzuki, 1996] and presents an efficient depth-first branch-and-bound algorithm for learning Bayesian network structures, based on the minimum description length (MDL) principle, for a given (consistent) variable ordering. The algorithm exhaustively searches through all network structures and guarantees to find the network with the best MDL score. Preliminary experiments show that the algorithm is efficient, and that the time complexity grows slowly with the sample size. The algorithm is useful for empirically studying both the performance of suboptimal heuristic search algorithms and the adequacy of the MDL principle in learning Bayesian networks.

[1]  David Maxwell Chickering,et al.  Learning Bayesian Networks is , 1994 .

[2]  Gregory F. Cooper,et al.  The ALARM Monitoring System: A Case Study with two Probabilistic Inference Techniques for Belief Networks , 1989, AIME.

[3]  A. H. Murphy,et al.  Hailfinder: A Bayesian system for forecasting severe weather , 1996 .

[4]  Edward H. Herskovits,et al.  Computer-based probabilistic-network construction , 1992 .

[5]  Gregory F. Cooper,et al.  A Bayesian method for the induction of probabilistic networks from data , 1992, Machine-mediated learning.

[6]  Joe Suzuki,et al.  Learning Bayesian Belief Networks Based on the Minimum Description Length Principle: An Efficient Algorithm Using the B & B Technique , 1996, ICML.

[7]  Remco R. Bouckaert,et al.  Probalistic Network Construction Using the Minimum Description Length Principle , 1993, ECSQARU.

[8]  Thorsten von Eicken,et al.  技術解説 IEEE Computer , 1999 .

[9]  D. A. Bell,et al.  Information Theory and Reliable Communication , 1969 .

[10]  Lise Getoor,et al.  Efficient learning using constrained sufficient statistics , 1999, AISTATS.

[11]  Remco R. Bouckaert,et al.  Properties of Bayesian Belief Network Learning Algorithms , 1994, UAI.

[12]  Wray L. Buntine.  A Guide to the Literature on Learning Probabilistic Networks from Data , 1996, IEEE Trans. Knowl. Data Eng..

[13]  Brent Boerlage.  Link Strength in Bayesian Networks , 1994 .

[14]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[15]  Jaswinder Pal Singh,et al.  Parallel Implementations of Probabilistic Inference , 1996, Computer.

[16]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.