Wavelet Shrinkage: Asymptopia?

Much recent effort has sought asymptotically minimax methods for recovering infinite dimensional objects-curves, densities, spectral densities, images-from noisy data. A now rich and complex body of work develops nearly or exactly minimax estimators for an array of interesting problems. Unfortunately, the results have rarely moved into practice, for a variety of reasons-among them being similarity to known methods, computational intractability and lack of spatial adaptivity. We discuss a method for curve estimation based on n noisy data: translate the empirical wavelet coefficients towards the origin by an amount √(2 log n) /√n. The proposal differs from those in current use, is computationally practical and is spatially adaptive; it thus avoids several of the previous objections. Further, the method is nearly minimax both for a wide variety of loss functions-pointwise error, global error measured in L p -norms, pointwise and global error in estimation of derivatives-and for a wide range of smoothness classes, including standard Holder and Sobolev classes, and bounded variation. This is a much broader near optimality than anything previously proposed: we draw loose parallels with near optimality in robustness and also with the broad near eigenfunction properties of wavelets themselves. Finally, the theory underlying the method is interesting, as it exploits a correspondence between statistical questions and questions of optimal recovery and information-based complexity

[1]  F. J. Anscombe,et al.  THE TRANSFORMATION OF POISSON, BINOMIAL AND NEGATIVE-BINOMIAL DATA , 1948 .

[2]  J. Wolfowitz Minimax Estimates of the Mean of a Normal Distribution with Known Variance , 1950 .

[3]  R. H. Farrell On the Lack of a Uniformly Consistent Sequence of Estimators of a Density Function in Certain Cases , 1967 .

[4]  J. Peetre New thoughts on Besov spaces , 1976 .

[5]  Charles A. Micchelli,et al.  A Survey of Optimal Recovery , 1977 .

[6]  L. Breiman,et al.  Variable Kernel Estimates of Multivariate Densities , 1977 .

[7]  I. Good,et al.  Density Estimation and Bump-Hunting by the Penalized Likelihood Method Exemplified by Scattering and Meteorite Data , 1980 .

[8]  Jerome Sacks,et al.  ASYMPTOTICALLY OPTIMUM KERNELS FOR DENSITY ESTIMATION AT A POINT , 1981 .

[9]  Derick Wood,et al.  Completeness of Context-Free Grammar Forms , 1981, J. Comput. Syst. Sci..

[10]  C. J. Stone,et al.  Optimal Global Rates of Convergence for Nonparametric Regression , 1982 .

[11]  C. J. Stone,et al.  OPTIMAL GLOBAL RATES OF CONVERGENCE FOR NONPARAMETRIC ESTIMATORS , 1982 .

[12]  I. A. Ibragimov,et al.  Bounds for the Risks of Non-Parametric Regression Estimates , 1982 .

[13]  M. R. Leadbetter,et al.  Extremes and Related Properties of Random Sequences and Processes: Springer Series in Statistics , 1983 .

[14]  Lucien Birgé Approximation dans les espaces métriques et théorie de l'estimation , 1983 .

[15]  M. Nussbaum Spline Smoothing in Regression Models and Asymptotic Efficiency in $L_2$ , 1985 .

[16]  P. Speckman Spline Smoothing and Optimal Rates of Convergence in Nonparametric Regression Models , 1985 .

[17]  Nonasymptotic minimax risk for Hellinger balls , 1985 .

[18]  S. Geer A New Approach to Least-Squares Estimation, with Applications , 1986 .

[19]  Henryk Wozniakowski,et al.  Information-based complexity , 1987, Nature.

[20]  H. Müller,et al.  Variable Bandwidth Kernel Estimators of Regression Curves , 1987 .

[21]  A. Nemirovskii,et al.  Some Problems on Nonparametric Estimation in Gaussian White Noise , 1987 .

[22]  D. Donoho One-sided inference about functionals of a density , 1988 .

[23]  R. DeVore,et al.  Interpolation of Besov-Spaces , 1988 .

[24]  L. Birge,et al.  The Grenader Estimator: A Nonasymptotic Approach , 1989 .

[25]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[26]  J. Friedman,et al.  FLEXIBLE PARSIMONIOUS SMOOTHING AND ADDITIVE MODELING , 1989 .

[27]  Bernard W. Silverman,et al.  Speed of Estimation in Positron Emission Tomography and Related Inverse Problems , 1990 .

[28]  D. Donoho,et al.  Minimax Risk Over Hyperrectangles, and Implications , 1990 .

[29]  O. Lepskii On a Problem of Adaptive Estimation in Gaussian White Noise , 1991 .

[30]  G. Weiss,et al.  Littlewood-Paley Theory and the Study of Function Spaces , 1991 .

[31]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[32]  David L. Donoho,et al.  Interpolating Wavelet Transforms , 1992 .

[33]  G. Kerkyacharian,et al.  Density estimation in Besov spaces , 1992 .

[34]  Iain M. Johnstone,et al.  Estimation d'une densité de probabilité par méthode d'ondelettes , 1992 .

[35]  A. Samarov Lower bound for the integral risk of density function estimates , 1992 .

[36]  R. DeVore,et al.  Fast wavelet techniques for near-optimal image processing , 1992, MILCOM 92 Conference Record.

[37]  Stéphane Mallat,et al.  Matching pursuits with time-frequency dictionaries , 1993, IEEE Trans. Signal Process..

[38]  T. Gasser,et al.  Locally Adaptive Bandwidth Choice for Kernel Regression Estimators , 1993 .

[39]  I. Daubechies,et al.  Multiresolution analysis, wavelets and fast algorithms on an interval , 1993 .

[40]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[41]  D. L. Donoho,et al.  Ideal spacial adaptation via wavelet shrinkage , 1994 .

[42]  D. Donoho Asymptotic minimax risk for sup-norm loss: Solution via optimal recovery , 1994 .

[43]  I. Johnstone Minimax Bayes, Asymptotic Minimax and Sparse Wavelet Priors , 1994 .

[44]  D. Donoho Statistical Estimation and Optimal Recovery , 1994 .

[45]  D. Donoho Nonlinear Solution of Linear Inverse Problems by Wavelet–Vaguelette Decomposition , 1995 .

[46]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[47]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[48]  L. Brown,et al.  Asymptotic equivalence of nonparametric regression and white noise , 1996 .

[49]  L. Brown,et al.  A constrained risk inequality with applications to nonparametric functional estimation , 1996 .