Variational Heteroscedastic Gaussian Process Regression

Standard Gaussian processes (GPs) model observations' noise as constant throughout input space. This is often a too restrictive assumption, but one that is needed for GP inference to be tractable. In this work we present a non-standard variational approximation that allows accurate inference in heteroscedastic GPs (i.e., under input-dependent noise conditions). Computational cost is roughly twice that of the standard GP, and also scales as O(n3). Accuracy is verified by comparing with the golden standard MCMC and its effectiveness is illustrated on several synthetic and real datasets of diverse characteristics. An application to volatility forecasting is also considered.

[1]  B. Silverman,et al.  Some Aspects of the Spline Smoothing Approach to Non‐Parametric Regression Curve Fitting , 1985 .

[2]  Charles G. Renfro,et al.  Benchmarks and software standards: A case study of GARCH procedures , 1998 .

[3]  Michael I. Jordan,et al.  Regression with input-dependent noise: A Gaussian process treatment , 1998 .

[4]  Chris Brooks,et al.  Benchmarks and the accuracy of GARCH model estimation , 2001 .

[5]  P. Hansen,et al.  Technical Appendix A Forecast Comparison of Volatility Models: Does Anything Beat a GARCH(1,1)? , 2005 .

[6]  Jun S. Liu,et al.  Monte Carlo strategies in scientific computing , 2001 .

[7]  Christopher K. I. Williams,et al.  Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning) , 2005 .

[8]  Gavin C. Cawley,et al.  Estimating Predictive Variances with Kernel Ridge Regression , 2005, MLCW.

[9]  Wolfram Burgard,et al.  Most likely heteroscedastic Gaussian process regression , 2007, ICML '07.

[10]  C. Rasmussen,et al.  Approximations for Binary Gaussian Process Classification , 2008 .

[11]  Kristian Kersting,et al.  Kernel Conditional Quantile Estimation via Reduction Revisited , 2009, 2009 Ninth IEEE International Conference on Data Mining.

[12]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[13]  Manfred Opper,et al.  The Variational Gaussian Approximation Revisited , 2009, Neural Computation.

[14]  Ryan P. Adams,et al.  Elliptical slice sampling , 2009, AISTATS.

[15]  Andrew Gordon Wilson,et al.  Copula Processes , 2010, NIPS.

[16]  Juha Karhunen,et al.  Approximate Riemannian Conjugate Gradient Learning for Fixed-Form Variational Bayes , 2010, J. Mach. Learn. Res..

[17]  C. Brownlees,et al.  A Practical Guide to Volatility Forecasting through Calm and Storm , 2011 .

[18]  M. Girolami,et al.  Riemann manifold Langevin and Hamiltonian Monte Carlo methods , 2011, Journal of the Royal Statistical Society: Series B (Statistical Methodology).