论文信息 - On the Concentration of Expectation and Approximate Inference in Layered Networks

On the Concentration of Expectation and Approximate Inference in Layered Networks

We present an analysis of concentration-of-expectation phenomena in layered Bayesian networks that use generalized linear models as the local conditional probabilities. This framework encompasses a wide variety of probability distributions, including both discrete and continuous random variables. We utilize ideas from large deviation analysis and the delta method to devise and evaluate a class of approximate inference algorithms for layered Bayesian networks that have superior asymptotic error bounds and very fast computation time.

Michael I. Jordan | XuanLong Nguyen

[1] Lawrence D. Brown. Fundamentals of Statistical Exponential Families , 1987 .

[2] Eric R. Ziegel,et al. Generalized Linear Models , 2002, Technometrics.

[3] Lawrence K. Saul,et al. Inference in Multilayer Networks via Large Deviation Bounds , 1998, NIPS.

[4] L. Brown. Fundamentals of statistical exponential families: with applications in statistical decision theory , 1986 .

[5] T. Plefka. Convergence condition of the TAP equation for the infinite-ranged Ising spin glass model , 1982 .

[6] Tom Minka,et al. Expectation Propagation for approximate Bayesian inference , 2001, UAI.

[7] W. Freeman,et al. Generalized Belief Propagation , 2000, NIPS.

[8] David Barber,et al. Gaussian Fields for Approximate Inference in Layered Sigmoid Belief Networks , 1999, NIPS.

[9] David Barber,et al. Variational Cumulant Expansions for Intractable Distributions , 2011, J. Artif. Intell. Res..

[10] Michael I. Jordan,et al. An Introduction to Variational Methods for Graphical Models , 1999, Machine-mediated learning.

[11] David Heckerman,et al. A Tractable Inference Algorithm for Diagnosing Multiple Diseases , 2013, UAI.

[12] Lawrence K. Saul,et al. Large Deviation Methods for Approximate Probabilistic Inference , 1998, UAI.