Efficient Bayesian inference methods via convex optimization and optimal transport

In this paper, we consider many problems in Bayesian inference - from drawing samples to posteriors, to calculating confidence intervals, to implementing posterior matching algorithms, by finding maps that push one distribution to another. We show that for a large class of problems (with log-concave likelihoods and log-concave priors), these problems can be efficiently solved using convex optimization. We provide example applications within the context of dynamic statistical signal processing.

[1]  Mark A. Girolami,et al.  Markov Chain Monte Carlo Methods for State-Space Models with Point Process Observations , 2012, Neural Computation.

[2]  Youssef M. Marzouk,et al.  Bayesian inference with optimal maps , 2011, J. Comput. Phys..

[3]  Rui Ma,et al.  Generalizing the Posterior Matching Scheme to higher dimensions via optimal transportation , 2011, 2011 49th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[4]  Emery N. Brown,et al.  Estimating a State-space Model from Point Process Observations Emery N. Brown , 2022 .

[5]  George Casella,et al.  Estimating bayesian credible intervals , 2003 .

[6]  Neri Merhav,et al.  Universal Prediction , 1998, IEEE Trans. Inf. Theory.

[7]  Jorma Rissanen,et al.  Minimum Description Length Principle , 2010, Encyclopedia of Machine Learning.

[8]  Matthew A. Wilson,et al.  Variational Bayesian inference for point process generalized linear models in neural spike trains analysis , 2010, 2010 IEEE International Conference on Acoustics, Speech and Signal Processing.

[9]  Todd P. Coleman,et al.  Estimating the directed information to infer causal relationships in ensemble neural spike train recordings , 2010, Journal of Computational Neuroscience.

[10]  Tim Hesterberg,et al.  Monte Carlo Strategies in Scientific Computing , 2002, Technometrics.

[11]  Abdullah Akce,et al.  A Feedback Information-Theoretic Approach to the Design of Brain–Computer Interfaces , 2010, Int. J. Hum. Comput. Interact..

[12]  C. Villani.  Topics in Optimal Transportation , 2003 .

[13]  Gábor Lugosi,et al.  Prediction, learning, and games , 2006 .

[14]  C. Granger.  Investigating causal relations by econometric models and cross-spectral methods , 1969 .

[15]  Haim H. Permuter,et al.  Universal estimation of directed information via sequential probability assignments , 2012, 2012 IEEE International Symposium on Information Theory Proceedings.

[16]  Emery N. Brown,et al.  Dynamic Analysis of Neural Encoding by Point Process Adaptive Filtering , 2004, Neural Computation.

[17]  Emery N. Brown,et al.  A Granger Causality Measure for Point Process Models of Ensemble Neural Spiking Activity , 2011, PLoS Comput. Biol..

[18]  Meir Feder,et al.  Optimal Feedback Communication Via Posterior Matching , 2009, IEEE Transactions on Information Theory.

[19]  Stephen P. Boyd,et al.  Convex Optimization , 2004, Algorithms and Theory of Computation Handbook.