This note describes power EP, an extension of Expectation Propagation (EP) that makes the computations more tractable. In this way, power EP is applicable to a wide variety of models, much more than EP. Instead of minimizing KL-divergence at each step, power EP minimizes α-divergence. This minimization turns out to be equivalent to minimizing KL-divergence with the exact distribution raised to a power. By choosing this power to cancel exponents, the problem may be substantially simplified. The resulting approximation is not the same as regular EP, but in practice is still very good, and allows tackling problems which are intractable under regular EP.
[1]
Shun-ichi Amari,et al.
Differential-geometrical methods in statistics
,
1985
.
[2]
Tom Minka,et al.
Expectation Propagation for approximate Bayesian inference
,
2001,
UAI.
[3]
Thomas P. Minka,et al.
The EP energy function and minimization schemes
,
2001
.
[4]
M. Trottini,et al.
A generalized predictive criterion for model selection
,
2002
.
[5]
Tom Minka,et al.
Expectation-Propogation for the Generative Aspect Model
,
2002,
UAI.
[6]
Tom Heskes,et al.
Fractional Belief Propagation
,
2002,
NIPS.
[7]
Charles M. Bishop,et al.
Variational Message Passing
,
2005,
J. Mach. Learn. Res..