Approximability of Probability Distributions

We consider the question of how well a given distribution can be approximated with probabilistic graphical models. We introduce a new parameter, effective treewidth, that captures the degree of approximability as a tradeoff between the accuracy and the complexity of approximation. We present a simple approach to analyzing achievable tradeoffs that exploits the threshold behavior of monotone graph properties, and provide experimental results that support the approach.

[1]  C. N. Liu,et al.  Approximating discrete probability distributions with dependence trees , 1968, IEEE Trans. Inf. Theory.

[2]  Judea Pearl,et al.  Probabilistic reasoning in intelligent systems - networks of plausible inference , 1991, Morgan Kaufmann series in representation and reasoning.

[3]  Frank Jensen,et al.  Optimal junction Trees , 1994, UAI.

[4]  Gregory F. Cooper,et al.  The Computational Complexity of Probabilistic Inference Using Bayesian Belief Networks , 1990, Artif. Intell..

[5]  B. Bollobás.  The evolution of random graphs , 1984 .

[6]  M. Birkner,et al.  Blow-up of semilinear PDE's at the critical dimension. A probabilistic approach , 2002 .

[7]  Béla Bollobás,et al.  The Structure of Random Graph Orders , 1997, SIAM J. Discret. Math..

[8]  Rina Dechter,et al.  Bucket elimination: A unifying framework for probabilistic inference , 1996, UAI.

[9]  Paul Erdös,et al.  On the Maximal Number of Strongly Independent Vertices in a Random Acyclic Directed Graph , 1984 .

[10]  Klaus-Uwe Höffgen,et al.  Learning and robust learning of product distributions , 1993, COLT '93.

[11]  P. Erdos,et al.  On the evolution of random graphs , 1984 .

[12]  Alina Beygelzimer,et al.  Inference Complexity as a Model-Selection Criterion for Learning Bayesian Networks , 2002, KR.

[13]  Moni Naor,et al.  Small-bias probability spaces: efficient constructions and applications , 1990, STOC '90.

[14]  G. Kalai,et al.  Every monotone graph property has a sharp threshold , 1996 .

[15]  A. Rbnyi.  ON THE EVOLUTION OF RANDOM GRAPHS , 2001 .

[16]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[17]  Nathan Srebro,et al.  Maximum likelihood bounded tree-width Markov networks , 2001, Artif. Intell..