Preventing the Dirac disaster: Wavelet based density estimation

This paper addresses the problem of choosing the optimal number of basis functions in constructing wavelet series density estimators. It is well known thatprojection estimators tend to overfit the density if the number of basis functions in the orthogonal expansion is too large. In extreme cases the estimator is close to the Dirac function concentrated at the observations. We propose a roughness measure of wavelet estimators and establish a data driven method for determining the number of levels to be included in the estimate. Our method exploits the idea of using the Fisher information functional as a roughness measure. The method is demonstrated on simulated data.

[1]  B. Silverman,et al.  On the Estimation of a Probability Density Function by the Maximum Penalized Likelihood Method , 1982 .

[2]  G. Kerkyacharian,et al.  Density estimation by kernel and wavelets methods: Optimality of Besov spaces , 1993 .

[3]  H. D. Brunk.  Univariate density estimation by orthogonal series , 1978 .

[4]  Peter Hall,et al.  Formulae for Mean Integrated Squared Error of Nonlinear Wavelet-Based Density Estimators , 1995 .

[5]  R. Kronmal,et al.  The Estimation of Probability Densities and Cumulatives by Fourier Series Methods , 1968 .

[6]  Tom Leonard,et al.  A Bayesian method for histograms , 1973 .

[7]  I. Johnstone,et al.  Density estimation by wavelet thresholding , 1996 .

[8]  A. Habibi,et al.  Introduction to wavelets , 1995, Proceedings of MILCOM '95.

[9]  A. Izenman.  Recent Developments in Nonparametric Density Estimation , 1991 .

[10]  Chong Gu,et al.  Smoothing spline density estimation: theory , 1993 .

[11]  Carl Taswell,et al.  WavBox 4: A Software Toolbox for Wavelet Transforms and Adaptive Wavelet Packet Decompositions , 1995 .

[12]  A. Haar.  Zur Theorie der orthogonalen Funktionensysteme , 1910 .

[13]  Spiridon Penev,et al.  On non-negative wavelet-based density estimators , 1997 .

[14]  G. Kerkyacharian,et al.  Density estimation in Besov spaces , 1992 .

[15]  P. Bickel.  Minimax Estimation of the Mean of a Normal Distribution when the Parameter Space is Restricted , 1981 .

[16]  Peter J. Huber,et al.  Fisher Information and Spline Interpolation , 1974 .

[17]  Donald L. Kreider,et al.  An introduction to linear analysis , 1966 .

[18]  Y. Meyer.  Wavelets and Operators , 1993 .

[19]  M. Wand,et al.  EXACT MEAN INTEGRATED SQUARED ERROR , 1992 .

[20]  A. Izenman.  Review Papers: Recent Developments in Nonparametric Density Estimation , 1991 .

[21]  G. Wahba.  Data-Based Optimal Smoothing of Orthogonal Series Density Estimates , 1981 .

[22]  R. A. Gaskins,et al.  Nonparametric roughness penalties for probability densities , 2022 .

[23]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  G. Walter.  Approximation of the delta function by wavelets , 1992 .

[25]  G. Beylkin.  On the representation of operators in bases of compactly supported wavelets , 1992 .

[26]  I. Daubechies.  Orthonormal bases of compactly supported wavelets , 1988 .

[27]  G. Walter.  Wavelets and other orthogonal systems with applications , 1994 .

[28]  Brani Vidakovic,et al.  Estimating the square root of a density via compactly supported wavelets , 1997 .