A conditional game for comparing approximations

We present a \conditional game" to be played between two approximate inference algorithms. We prove that exact inference is an optimal strategy and demonstrate how the game can be used to estimate the relative accuracy of two dierent approximations in the absence of exact marginals.

[1]  Bruno de Finetti,et al.  Probabilism , 1989 .

[2]  X. Jin.  Factor graphs and the Sum-Product Algorithm , 2002 .

[3]  Michael L. Littman,et al.  Graphical Models for Game Theory , 2001, UAI.

[4]  Martin J. Wainwright,et al.  Tree-reweighted belief propagation algorithms and approximate ML estimation by pseudo-moment matching , 2003, AISTATS.

[5]  Yee Whye Teh,et al.  Structured Region Graphs: Morphing EP into GBP , 2005, UAI.

[6]  Flemming Topsøe,et al.  Information-theoretical optimization techniques , 1979, Kybernetika.

[7]  Hilbert J. Kappen,et al.  Bounds on marginal probability distributions , 2008, NIPS.

[8]  J. Propp,et al.  Exact sampling with coupled Markov chains and applications to statistical mechanics , 1996 .

[9]  Tom Minka,et al.  Expectation Propagation for approximate Bayesian inference , 2001, UAI.

[10]  Yuan Qi,et al.  Tree-structured Approximations by Expectation Propagation , 2003, NIPS.

[11]  Alan L. Yuille,et al.  The Concave-Convex Procedure , 2003, Neural Computation.

[12]  Adnan Darwiche,et al.  A differential approach to inference in Bayesian networks , 2000, JACM.

[13]  Jeff Bilmes.  UAI 06 Inference Evaluation Results , 2006 .

[14]  Pedro M. Domingos,et al.  Approximate Inference by Compilation to Arithmetic Circuits , 2010, NIPS.

[15]  Lothar M. Schmitt,et al.  Theory of Genetic Algorithms II: models for genetic operators over the string-tensor representation of populations and convergence to global optima for arbitrary fitness function under scaling , 2004, Theor. Comput. Sci..

[16]  Zoubin Ghahramani,et al.  Choosing a Variable to Clamp: Approximate Inference Using Conditioned Belief Propagation , 2009 .

[17]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[18]  Committee Chair.  Fixing and Extending the Multiplicative Approximation Scheme , 2009 .

[19]  Hilbert J. Kappen,et al.  Loop Corrected Belief Propagation , 2007, AISTATS.

[20]  Joris M. Mooij,et al.  libDAI: A Free and Open Source C++ Library for Discrete Approximate Inference in Graphical Models , 2010, J. Mach. Learn. Res..

[21]  W. Freeman,et al.  Generalized Belief Propagation , 2000, NIPS.

[22]  Yoav Freund,et al.  Boosting a weak learning algorithm by majority , 1990, COLT '90.

[23]  Adam Tauman Kalai,et al.  Dueling algorithms , 2011, STOC '11.

[24]  Hilbert J. Kappen,et al.  Approximate Inference and Constrained Optimization , 2002, UAI.

[25]  Judea Pearl,et al.  Fusion, Propagation, and Structuring in Belief Networks , 1986, Artif. Intell..