Two Useful Bounds for Variational Inference

We review and derive two lower bounds on the expectation of the log-sum function in the context of variational inference. The first bound relies on the first-order Taylor expansion about an auxiliary parameter. The second bound relies on an auxiliary probability distribution. We show how these auxiliary parameters can be removed entirely from the model. We then discuss the advantage of keeping these parameters in certain cases, giving an example of a likelihood/prior pair for each case.