On the condition number anomaly of Gaussian correlation matrices

Abstract Spatial correlation matrices appear in a large variety of applications. For example, they are an essential component of spatial Gaussian processes, also known as spatial linear models or Kriging estimators, which are powerful and well-established tools for a multitude of engineering applications such as the design and analysis of computer experiments, geostatistical problems and meteorological tasks. In radial basis function interpolation, Gaussian correlation matrices arise frequently as interpolation matrices from the Gaussian radial kernel function. In the field of data assimilation in numerical weather prediction, such matrices arise as background error covariances. Over the past thirty years, it was observed by several authors from several fields that the Gaussian correlation model is exceptionally prone to suffer from ill-conditioning, but a quantitative theoretical explanation for this anomaly was lacking. In this paper, a proof for the special position of the Gaussian correlation matrix is given. The theoretical findings are illustrated by numerical experiment.

[1]  P. Lancaster.  On eigenvalues of matrices dependent on a parameter , 1964 .

[2]  Wei-Liem Loh,et al.  Estimating structured correlation matrices in smooth Gaussian random field models , 2000 .

[3]  D. J. Hartfiel,et al.  Dense sets of diagonalizable matrices , 1995 .

[4]  Nancy Nichols,et al.  Conditioning of incremental variational data assimilation, with application to the Met Office system , 2011 .

[5]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[6]  Ralf Zimmermann,et al.  Asymptotic Behavior of the Likelihood Function of Covariance Matrices of Spatial Gaussian Processes , 2010, J. Appl. Math..

[7]  G. Stewart.  Error and Perturbation Bounds for Subspaces Associated with Certain Eigenvalue Problems , 1973 .

[8]  P. Michor,et al.  Choosing roots of polynomials smoothly , 1998, math/9801026.

[9]  Mike Rees,et al.  5. Statistics for Spatial Data , 1993 .

[10]  Noel A. C. Cressie,et al.  Statistics for Spatial Data: Cressie/Statistics , 1993 .

[11]  Alan C. Koivunen,et al.  On the condition number of Gaussian sample-covariance matrices , 2000, IEEE Trans. Geosci. Remote. Sens..

[12]  Kanti V. Mardia,et al.  On multimodality of the likelihood in the spatial linear model , 1989 .

[13]  Andy J. Keane,et al.  Engineering Design via Surrogate Modelling - A Practical Guide , 2008 .

[14]  K. Mardia,et al.  Maximum likelihood estimation of models for residual covariance in spatial regression , 1984 .

[15]  Seung-Jean Kim,et al.  Condition‐number‐regularized covariance estimation , 2013, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[16]  R. Ababou,et al.  On the condition number of covariance matrices in kriging, estimation, and simulation of random fields , 1994 .

[17]  B. Baxter,et al.  The Interpolation Theory of Radial Basis Functions , 2010, 1006.2443.

[18]  Phil Diamond,et al.  Robustness of variograms and conditioning of kriging matrices , 1984 .

[19]  D. Posa.  Conditioning of the stationary kriging matrices for some well-known covariance models , 1989 .

[20]  Pritam Ranjan,et al.  A Computationally Stable Approach to Gaussian Process Interpolation of Deterministic Computer Simulation Data , 2010, Technometrics.

[21]  Andrew D. Back,et al.  Radial Basis Functions , 2001 .

[22]  Robert Haining,et al.  Statistics for spatial data: by Noel Cressie, 1991, John Wiley & Sons, New York, 900 p., ISBN 0-471-84336-9, US $89.95 , 1993 .

[23]  N. Nichols,et al.  Conditioning of the 3DVAR Data Assimilation Problem , 2009 .

[24]  A. C. Koivunen,et al.  The Feasibility of Data Whitening to Improve Performance of Weather Radar , 1999 .

[25]  I. J. Schoenberg,et al.  Metric spaces and positive definite functions , 1938 .

[26]  Matthew M Lin,et al.  On the Nonnegative Rank of Euclidean Distance Matrices. , 2010, Linear algebra and its applications.

[27]  Peter Challenor,et al.  Computational Statistics and Data Analysis the Effect of the Nugget on Gaussian Process Emulators of Computer Models , 2022 .

[28]  H. G. ter Morsche,et al.  Computation of eigenvalue and eigenvector derivatives for a general complex-valued eigensystem , 2006 .

[29]  J. Gower.  Properties of Euclidean and non-Euclidean distance matrices , 1985 .

[30]  Max D. Morris,et al.  Six factors which affect the condition number of matrices associated with kriging , 1997 .

[31]  A. Yaglom.  Correlation Theory of Stationary and Related Random Functions I: Basic Results , 1987 .

[32]  I. J. Schoenberg.  Remarks to Maurice Frechet's Article ``Sur La Definition Axiomatique D'Une Classe D'Espace Distances Vectoriellement Applicable Sur L'Espace De Hilbert , 1935 .

[33]  Brian D. Ripley,et al.  Problems with likelihood estimation of covariance functions of spatial Gaussian processes , 1987 .

[34]  F. Rellich,et al.  Störungstheorie der Spektralzerlegung , 1937 .