Invariances of random fields paths, with applications in Gaussian Process Regression

We study pathwise invariances of centred random fields that can be controlled through the covariance. A result involving composition operators is obtained in second-order settings, and we show that various path properties including additivity boil down to invariances of the covariance kernel. These results are extended to a broader class of operators in the Gaussian case, via the Loeve isometry. Several covariance-driven pathwise invariances are illustrated, including fields with symmetric paths, centred paths, harmonic paths, or sparse paths. The proposed approach delivers a number of promising results and perspectives in Gaussian process regression.

[1]  J. Mercer.  Functions of Positive and Negative Type, and their Connection with the Theory of Integral Equations , 1909 .

[2]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[3]  G. Matheron.  Principles of geostatistics , 1963 .

[4]  Malcolm R Leadbetter,et al.  Stationary and Related Stochastic Processes: Sample Function Properties and Their Applications , 1967 .

[5]  T. T. Kadota,et al.  Differentiation of Karhunen-Loève expansion and application to optimum reception of sure signals in noise , 1967, IEEE Trans. Inf. Theory.

[6]  J. Kuelbs,et al.  Expansions of vectors in a Banach space related to Gaussian measures. , 1971 .

[7]  K. Parthasarathy,et al.  Positive Definite Kernels, Continuous Tensor Products, and Central Limit Theorems of Probability Theory , 1972 .

[8]  Miss A.O. Penney.  (b) , 1974, The New Yale Book of Quotations.

[9]  Srishti D. Chatterji,et al.  Positive Definite Kernels , 1983 .

[10]  C. J. Stone,et al.  Additive Regression and Other Nonparametric Models , 1985 .

[11]  A. Pietsch.  Eigenvalue distribution of compact operators , 1986 .

[12]  R. Adler.  An introduction to continuity, extrema, and related topics for general Gaussian processes , 1990 .

[13]  M. Yor,et al.  Continuous martingales and Brownian motion , 1990 .

[14]  Henry P. Wynn,et al.  Screening, predicting, and computer experiments , 1992 .

[15]  R. K. Singh,et al.  Composition operators on function spaces , 1993 .

[16]  Martin Brown,et al.  SUPANOVA: a sparse, transparent modelling approach , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[17]  Roger Woodard,et al.  Interpolation of Spatial Data: Some Theory for Kriging , 1999, Technometrics.

[18]  Donald R. Jones,et al.  A Taxonomy of Global Optimization Methods Based on Response Surfaces , 2001, J. Glob. Optim..

[19]  Ilya M. Sobol,et al.  Theorems and examples on high dimensional model representation , 2003, Reliab. Eng. Syst. Saf..

[20]  A. Berlinet,et al.  Reproducing kernel Hilbert spaces in probability and statistics , 2004 .

[21]  A. OHagan,et al.  Bayesian analysis of computer code outputs: A tutorial , 2006, Reliab. Eng. Syst. Saf..

[22]  P. Deheuvels.  A Karhunen-Loeve expansion for a mean-centered Brownian bridge , 2007 .

[23]  Rhodri Hayward,et al.  Screening , 2008, The Lancet.

[24]  Van Der Vaart,et al.  Rates of contraction of posterior distributions based on Gaussian process priors , 2008 .

[25]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[26]  S. Geer,et al.  High-dimensional additive modeling , 2008, 0806.4115.

[27]  Robert Schaback,et al.  Solving the Laplace equation by meshless collocation using harmonic kernels , 2009, Adv. Comput. Math..

[28]  Francis R. Bach,et al.  High-Dimensional Non-Linear Variable Selection through Hierarchical Kernel Learning , 2009, ArXiv.

[29]  M. Scheuerer.  Regularity of the sample paths of a general second order random field , 2010 .

[30]  Sonja Kuhnt,et al.  Design and analysis of computer experiments , 2010 .

[31]  Henryk Wozniakowski,et al.  On decompositions of multivariate functions , 2009, Math. Comput..

[32]  Michael Scheuerer,et al.  A Comparison of Models and Methods for Spatial Interpolation in Statistics and Numerical Analysis , 2010 .

[33]  Yu. I. Ingster,et al.  Detection of sparse additive functions , 2010, 1011.6369.

[34]  Carl E. Rasmussen,et al.  Additive Gaussian Processes , 2011, NIPS.

[35]  Harry van Zanten,et al.  Information Rates of Nonparametric Gaussian Process Methods , 2011, J. Mach. Learn. Res..

[36]  D. Ginsbourger,et al.  Additive Covariance Kernels for High-Dimensional Gaussian Process Modeling , 2011, 1111.6233.

[37]  Ingo Steinwart,et al.  Mercer’s Theorem on General Domains: On the Interaction between Measures, Kernels, and RKHSs , 2012 .

[38]  D. Ginsbourger,et al.  Argumentwise invariant kernels for the approximation of invariant functions , 2012 .

[39]  Martin J. Wainwright,et al.  Minimax-Optimal Rates For Sparse Additive Models Over Kernel Classes Via Convex Programming , 2010, J. Mach. Learn. Res..

[40]  Michael Scheuerer,et al.  Covariance Models for Divergence-Free and Curl-Free Random Vector Fields , 2012 .

[41]  Robert Schaback,et al.  Solving the 3D Laplace equation by meshless collocation via harmonic kernels , 2013, Adv. Comput. Math..

[42]  Jin V. Liu.  Karhunen–Loève expansion for additive Brownian motions , 2013 .

[43]  R. Tibshirani,et al.  Generalized Additive Models , 1986 .

[44]  Steen Pedersen,et al.  Introduction to Continuity , 2015 .