Sparse Approximations for Non-Conjugate Gaussian Process Regression

Notes: This report only shows some preliminary work on scaling Gaussian process models that use non-Gaussian likelihoods. As there are recently arxived papers on the similar idea [1,2], this report will stay as is, please consult the two papers above for a proper discussion and experiments.

[1]  Robert Price,et al.  A useful theorem for nonlinear devices having Gaussian inputs , 1958, IRE Trans. Inf. Theory.

[2]  G. Bonnet.  Transformations des signaux aléatoires a travers les systèmes non linéaires sans mémoire , 1964 .

[3]  D. Böhning.  Multinomial logistic regression algorithm , 1992 .

[4]  Michael I. Jordan,et al.  Bayesian parameter estimation via variational methods , 2000, Stat. Comput..

[5]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[6]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[7]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[8]  John D. Lafferty,et al.  Correlated Topic Models , 2005, NIPS.

[9]  Guillaume Bouchard.  Efficient Bounds for the Softmax Function and Applications to Approximate Inference in Hybrid models , 2008 .

[10]  Hugh F. Durrant-Whyte,et al.  On entropy approximation for Gaussian mixture random vectors , 2008, 2008 IEEE International Conference on Multisensor Fusion and Integration for Intelligent Systems.

[11]  Michalis K. Titsias,et al.  Variational Learning of Inducing Variables in Sparse Gaussian Processes , 2009, AISTATS.

[12]  Mohammad Emtiyaz Khan,et al.  Piecewise Bounds for Estimating Bernoulli-Logistic Latent Gaussian Models , 2011, ICML.

[13]  Mohammad Emtiyaz Khan,et al.  Fast Bayesian Inference for Non-Conjugate Gaussian Process Regression , 2012, NIPS.

[14]  David M. Blei,et al.  Nonparametric variational inference , 2012, ICML.

[15]  Mohammad Emtiyaz Khan,et al.  Variational learning for latent Gaussian model of discrete data , 2012 .

[16]  Miguel Lázaro-Gredilla,et al.  Doubly Stochastic Variational Bayes for non-Conjugate Inference , 2014, ICML.

[17]  Daan Wierstra,et al.  Stochastic Backpropagation and Approximate Inference in Deep Generative Models , 2014, ICML.

[18]  Sean Gerrish,et al.  Black Box Variational Inference , 2013, AISTATS.

[19]  Edwin V. Bonilla,et al.  Automated Variational Inference for Gaussian Process Models , 2014, NIPS.

[20]  Stephen J. Roberts,et al.  Variational Inference for Gaussian Process Modulated Poisson Processes , 2014, ICML.

[21]  James Hensman,et al.  Scalable Variational Gaussian Process Classification , 2014, AISTATS.

[22]  Max Welling,et al.  Markov Chain Monte Carlo and Variational Inference: Bridging the Gap , 2014, ICML.