Most likely heteroscedastic Gaussian process regression

This paper presents a novel Gaussian process (GP) approach to regression with input-dependent noise rates. We follow Goldberg et al.'s approach and model the noise variance using a second GP in addition to the GP governing the noise-free output value. In contrast to Goldberg et al., however, we do not use a Markov chain Monte Carlo method to approximate the posterior noise variance but a most likely noise approach. The resulting model is easy to implement and can directly be used in combination with various existing extensions of the standard GPs such as sparse approximations. Extensive experiments on both synthetic and real-world data, including a challenging perception problem in robotics, show the effectiveness of most likely heteroscedastic GP regression.

[1]  B. Silverman,et al.  Some Aspects of the Spline Smoothing Approach to Non‐Parametric Regression Curve Fitting , 1985 .

[2]  Yoshua Bengio,et al.  Convergence Properties of the K-Means Algorithms , 1994, NIPS.

[3]  M. Sigrist Air monitoring by spectroscopic techniques , 1994 .

[4]  P. M. Williams,et al.  Using Neural Networks to Model Conditional Multivariate Densities , 1996, Neural Computation.

[5]  Paul W. Goldberg,et al.  Regression with Input-dependent Noise: A Gaussian Process Treatment , 1997, NIPS.

[6]  Christopher K. I. Williams Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond , 1999, Learning in Graphical Models.

[7]  M. Wand,et al.  Kriging with Nonparametric Variance Function Estimation , 1999, Biometrics.

[8]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[9]  Sebastian Thrun,et al.  Probabilistic robotics , 2002, CACM.

[10]  Carl E. Rasmussen,et al.  Warped Gaussian Processes , 2003, NIPS.

[11]  M. Yuan,et al.  Doubly penalized likelihood estimator in heteroscedastic regression , 2004 .

[12]  Alexander J. Smola,et al.  Heteroscedastic Gaussian process regression , 2005, ICML.

[13]  Gavin C. Cawley,et al.  Estimating Predictive Variances with Kernel Ridge Regression , 2005, MLCW.

[14]  Zoubin Ghahramani,et al.  Variable Noise and Dimensionality Reduction for Sparse Gaussian processes , 2006, UAI.

[15]  David J. Nott,et al.  Semiparametric estimation of mean and variance functions for non-Gaussian data , 2006, Comput. Stat..

[16]  R. Kohn,et al.  Locally Adaptive Semiparametric Estimation of the Mean and Variance Functions in Regression Models , 2006 .

[17]  Stefan Schaal,et al.  Kernel Carpentry for Online Regression Using Randomly Varying Coefficient Model , 2007, IJCAI.

[18]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.