Learning Bayesian Nets that Perform Well

A Bayesian net (BN) is more than a succinct way to encode a probabilistic distribution; it also corresponds to a function used to answer queries. A BN can therefore be evaluated by the accuracy of the answers it returns. Many algorithms for learning BNs, however, attempt to optimize another criterion (usually likelihood, possibly augmented with a regularizing term), which is independent of the distribution of queries that are posed. This paper takes the "performance criteria" seriously, and considers the challenge of computing the BN whose performance -- read "accuracy over the distribution of queries" -- is optimal. We show that many aspects of this learning task are more difficult than the corresponding subtasks in the standard model.

[1]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[2]  W. Hoeffding.  Probability Inequalities for sums of Bounded Random Variables , 1963 .

[3]  Nir Friedman,et al.  On the Sample Complexity of Learning Bayesian Networks , 1996, UAI.

[4]  Michael Luby,et al.  Approximating Probabilistic Inference in Bayesian Belief Networks is NP-Hard , 1993, Artif. Intell..

[5]  Wai Lam,et al.  LEARNING BAYESIAN BELIEF NETWORKS: AN APPROACH BASED ON THE MDL PRINCIPLE , 1994, Comput. Intell..

[6]  Michael Kearns,et al.  Efficient noise-tolerant learning from statistical queries , 1993, STOC.

[7]  Dan Roth,et al.  On the Hardness of Approximate Reasoning , 1993, IJCAI.

[8]  Klaus-Uwe Höffgen,et al.  Learning and robust learning of product distributions , 1993, COLT '93.

[9]  Nir Friedman,et al.  Building Classifiers Using Bayesian Networks , 1996, AAAI/IAAI, Vol. 2.

[10]  Stuart J. Russell,et al.  Local Learning in Probabilistic Networks with Hidden Variables , 1995, IJCAI.

[11]  BuntineWray.  A Guide to the Literature on Learning Probabilistic Networks from Data , 1996 .

[12]  Tom M. Mitchell,et al.  A Model for Learning Systems , 1977, IJCAI.

[13]  Wray L. Buntine.  A Guide to the Literature on Learning Probabilistic Networks from Data , 1996, IEEE Trans. Knowl. Data Eng..

[14]  A. Hasman,et al.  Probabilistic reasoning in intelligent systems: Networks of plausible inference , 1991 .

[15]  Russell Greiner,et al.  PALO: A Probabilistic Hill-Climbing Algorithm , 1996, Artif. Intell..

[16]  Dan Roth,et al.  Learning to reason , 1994, JACM.