Predicting the Hardness of Learning Bayesian Networks

There are various algorithms for finding a Bayesian network structure (BNS) that is optimal with respect to a given scoring function. No single algorithm dominates the others in speed, and, given a problem instance, it is a priori unclear which algorithm will perform best and how fast it will solve the problem. Estimating the runtimes directly is extremely difficult as they are complicated functions of the instance. The main contribution of this paper is characterization of the empirical hardness of an instance for a given algorithm based on a novel collection of non-trivial, yet efficiently computable features. Our empirical results, based on the largest evaluation of state-of-the-art BNS learning algorithms to date, demonstrate that we can predict the runtimes to a reasonable degree of accuracy, and effectively select algorithms that perform well on a particular instance. Moreover, we also show how the results can be utilized in building a portfolio algorithm that combines several individual algorithms in an almost optimal manner.

[1]  Kevin Leyton-Brown,et al.  Algorithm Runtime Prediction: Methods and Evaluation (Extended Abstract) , 2015, IJCAI.

[2]  Kevin Leyton-Brown,et al.  SATzilla: Portfolio-based Algorithm Selection for SAT , 2008, J. Artif. Intell. Res..

[3]  Changhe Yuan,et al.  An Improved Admissible Heuristic for Learning Optimal Bayesian Networks , 2012, UAI.

[4]  Changhe Yuan,et al.  Learning Optimal Bayesian Networks: A Shortest Path Perspective , 2013, J. Artif. Intell. Res..

[5]  Luca Pulina,et al.  Treewidth: A Useful Marker of Empirical Hardness in Quantified Boolean Logic Encodings , 2008, LPAR.

[6]  James Cussens,et al.  Bayesian network learning with cutting planes , 2011, UAI.

[7]  Eugene Fink,et al.  How to Solve It Automatically: Selection Among Problem Solving Methods , 1998, AIPS.

[8]  Oren Etzioni,et al.  PRODIGY: an integrated architecture for planning and learning , 1991, SGAR.

[9]  Tommi S. Jaakkola,et al.  Learning Bayesian Network Structure using LP Relaxations , 2010, AISTATS.

[10]  Tomi Silander,et al.  A Simple Approach for Finding the Globally Optimal Bayesian Network Structure , 2006, UAI.

[11]  David Maxwell Chickering,et al.  A Bayesian Approach to Tackling Hard Computational Problems (Preliminary Report) , 2001, Electron. Notes Discret. Math..

[12]  Andrew W. Moore,et al.  Finding optimal Bayesian networks by dynamic programming , 2005 .

[13]  Qiang Ji,et al.  Efficient Structure Learning of Bayesian Networks using Constraints , 2011, J. Mach. Learn. Res..

[14]  Kevin Leyton-Brown,et al.  Algorithm runtime prediction: Methods & evaluation , 2012, Artif. Intell..

[15]  J. Ross Quinlan,et al.  Simplifying decision trees , 1987, Int. J. Hum. Comput. Stud..

[16]  David Maxwell Chickering,et al.  Learning Bayesian Networks is NP-Complete , 2016, AISTATS.

[17]  Yoav Shoham,et al.  Learning the Empirical Hardness of Optimization Problems: The Case of Combinatorial Auctions , 2002, CP.

[18]  James Cussens,et al.  Advances in Bayesian Network Learning using Integer Programming , 2013, UAI.

[19]  David Maxwell Chickering,et al.  Learning Bayesian Networks: The Combination of Knowledge and Statistical Data , 1994, Machine Learning.

[20]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[21]  Satoru Miyano,et al.  Finding Optimal Models for Small Gene Networks , 2003 .

[22]  Mikko Koivisto,et al.  Exact Bayesian Structure Discovery in Bayesian Networks , 2004, J. Mach. Learn. Res..

[23]  Richard E. Neapolitan,et al.  Learning Bayesian networks , 2007, KDD '07.

[24]  Eugene C. Freuder,et al.  Using CBR to Select Solution Strategies in Constraint Programming , 2005, ICCBR.

[25]  Michel Lemaître,et al.  Branch and Bound Algorithm Selection by Performance Prediction , 1998, AAAI/IAAI.

[26]  R. Geoff Dromey,et al.  An algorithm for the selection problem , 1986, Softw. Pract. Exp..

[27]  Changhe Yuan,et al.  Evaluating Anytime Algorithms for Learning Optimal Bayesian Networks , 2013, UAI.