Oracle inequalities for inverse problems

We consider a sequence space model of statistical linear inverse problems where we need to estimate a function f from indirect noisy observations. Let a finite set A of linear estimators be given. Our aim is to mimic the estimator in A that has the smallest risk on the true f. Under general conditions, we show that this can be achieved by simple minimization of an unbiased risk estimator, provided the singular values of the operator of the inverse problem decrease as a power law. The main result is a nonasymptotic oracle inequality that is shown to be asymptotically exact. This inequality can also be used to obtain sharp minimax adaptive results. In particular, we apply it to show that minimax adaptation on ellipsoids in the multivariate anisotropic case is realized by minimization of unbiased risk estimator without any loss of efficiency with respect to optimal nonadaptive procedures.

[1]  E. Stein,et al.  Introduction to Fourier Analysis on Euclidean Spaces. , 1971 .

[2]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[3]  G. Wahba.  Practical Approximate Solutions to Linear Operator Equations When the Data are Noisy , 1977 .

[4]  W. Härdle,et al.  Optimal Bandwidth Selection in Nonparametric Regression Function Estimation , 1985 .

[5]  Ker-Chau Li,et al.  Asymptotic optimality of CL and generalized cross-validation in ridge regression with application to spline smoothing , 1986 .

[6]  Ker-Chau Li,et al.  Asymptotic Optimality for $C_p, C_L$, Cross-Validation and Generalized Cross-Validation: Discrete Index Set , 1987 .

[7]  Grace Wahba,et al.  Spline Models for Observational Data , 1990 .

[8]  Bernard W. Silverman,et al.  Speed of Estimation in Positron Emission Tomography and Related Inverse Problems , 1990 .

[9]  Boris Polyak,et al.  Asymptotic Optimality of the $C_p$-Test for the Orthogonal Series Estimation of Regression , 1991 .

[10]  Ja-Yong Koo,et al.  Optimal Rates of Convergence for Nonparametric Statistical Inverse Problems , 1993 .

[11]  A. Tsybakov,et al.  Minimax theory of image reconstruction , 1993 .

[12]  D. L. Donoho,et al.  Ideal spacial adaptation via wavelet shrinkage , 1994 .

[13]  A. Kneip.  Ordered Linear Smoothers , 1994 .

[14]  D. Donoho.  Nonlinear Solution of Linear Inverse Problems by Wavelet–Vaguelette Decomposition , 1995 .

[15]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[16]  I. Johnstone,et al.  Neo-classical minimax problems, thresholding and adaptive function estimation , 1996 .

[17]  Bernard A. Mair,et al.  Statistical Inverse Estimation in Hilbert Scales , 1996, SIAM J. Appl. Math..

[18]  I. Johnstone.  WAVELET SHRINKAGE FOR CORRELATED DATA AND INVERSE PROBLEMS: ADAPTIVITY RESULTS , 1999 .

[19]  A. Goldenshluger,et al.  Adaptive estimation of linear functionals in Hilbert scales from indirect white noise observations , 2000 .

[20]  Alexander Goldenshluger,et al.  Adaptive Prediction and Estimation in Linear Regression with Infinitely Many Parameters , 2001 .

[21]  P. Massart,et al.  Gaussian model selection , 2001 .

[22]  G. Golubev,et al.  A statistical approach to the Cauchy problem for the Laplace equation , 2001 .

[23]  L. Birge,et al.  An alternative point of view on Lepski's method , 2001 .

[24]  A. Tsybakov,et al.  Sharp adaptation for inverse problems with random noise , 2002 .

[25]  A. Berlinet,et al.  Nonparametric Curve Estimation , 2004 .