Robust GCV choice of the regularization parameter for correlated data

We consider Tikhonov regularization of linear inverse problems with discrete noisy data containing correlated errors. Generalized cross-validation (GCV) is a prominent parameter choice method, but it is known to perform poorly if the sample size n is small or if the errors are correlated, sometimes giving the extreme value 0. We explain why this can occur and show that the robust GCV methods perform better. In particular, it is shown that, for any data set, there is a value of the robustness parameter below which the strong robust GCV method (R(1)GCV) will not choose the value 0. We also show that, if the errors are correlated with a certain covariance model, then, for a range of values of the unknown correlation parameter, the "expected" R(1)GCV estimate has a near optimal rate as a n -> infinity. Numerical results for the problem of second derivative estimation are consistent with the theoretical results and show that R(1)GCV gives reliable and accurate estimates.

[1]  Press,et al.  Class of fast methods for processing irregularly sampled or otherwise inhomogeneous one-dimensional data. , 1995, Physical review letters.

[2]  M. A. Lukas.  Comparisons of parameter choice methods for regularization with discrete noisy data , 1998 .

[3]  Chong Gu,et al.  Smoothing spline Gaussian regression: more scalable computation via efficient approximation , 2004 .

[4]  F. Bauer,et al.  Local Solutions to Inverse Problems in Geodesy , 2006 .

[5]  Chong Gu,et al.  Optimal Smoothing with Correlated Data , 2004 .

[6]  M. A. Lukas.  Robust generalized cross-validation for choosing the regularization parameter , 2006 .

[7]  D. Titterington,et al.  A cautionary note about crossvalidatory choice , 1989 .

[8]  M. A. Lukas.  Asymptotic optimality of generalized cross-validation for choosing the regularization parameter , 1993 .

[9]  Convergence rates for moment collocation solutions of linear operator equations , 1995 .

[10]  G. Wahba.  Practical Approximate Solutions to Linear Operator Equations When the Data are Noisy , 1977 .

[11]  Grace Wahba,et al.  Spline Models for Observational Data , 1990 .

[12]  Douglas Nychka,et al.  Confidence Intervals for Nonparametric Curve Estimates , 2001 .

[13]  Per Christian Hansen,et al.  Rank-Deficient and Discrete Ill-Posed Problems , 1996 .

[14]  R. Moyeed,et al.  Making robust the cross-validatory choice of smoothing parameter in spline smoothing regression , 1989 .

[15]  W. Wamsteker,et al.  Estimation of regularization parameters in multiple-image deblurring , 2004 .

[16]  D. S. Mitrinovic,et al.  Classical and New Inequalities in Analysis , 1992 .

[17]  Yuhong Yang,et al.  Nonparametric Regression with Correlated Errors , 2001 .

[18]  Ker-Chau Li,et al.  Asymptotic optimality of CL and generalized cross-validation in ridge regression with application to spline smoothing , 1986 .

[19]  Adhemar Bultheel,et al.  Multiple wavelet threshold estimation by generalized cross validation for images with correlated noise , 1999, IEEE Trans. Image Process..

[20]  Per Christian Hansen,et al.  REGULARIZATION TOOLS: A Matlab package for analysis and solution of discrete ill-posed problems , 1994, Numerical Algorithms.

[21]  Mark A. Lukas,et al.  Comparing parameter choice methods for regularization of ill-posed problems , 2011, Math. Comput. Simul..

[22]  Behavior near zero of the distribution of GCV smoothing parameter estimates , 1995 .

[23]  Samuel C. Kou,et al.  Smoothers and the Cp, Generalized Maximum Likelihood, and Extended Exponential Criteria , 2002 .

[24]  Peter Craven,et al.  Smoothing noisy data with spline functions , 1978 .

[25]  B. Efron.  Selection Criteria For Scatterplot Smoothers , 1999 .