Nonparametric density deconvolution by weighted kernel estimators

Nonparametric density estimation in the presence of measurement error is considered. The usual kernel deconvolution estimator seeks to account for the contamination in the data by employing a modified kernel. In this paper a new approach based on a weighted kernel density estimator is proposed. Theoretical motivation is provided by the existence of a weight vector that perfectly counteracts the bias in density estimation without generating an excessive increase in variance. In practice a data driven method of weight selection is required. Our strategy is to minimize the discrepancy between a standard kernel estimate from the contaminated data on the one hand, and the convolution of the weighted deconvolution estimate with the measurement error density on the other hand. We consider a direct implementation of this approach, in which the weights are optimized subject to sum and non-negativity constraints, and a regularized version in which the objective function includes a ridge-type penalty. Numerical tests suggest that the weighted kernel estimation can lead to tangible improvements in performance over the usual kernel deconvolution estimator. Furthermore, weighted kernel estimates are free from the problem of negative estimation in the tails that can occur when using modified kernels. The weighted kernel approach generalizes to the case of multivariate deconvolution density estimation in a very straightforward manner.

[1]  Javier M. Moguerza,et al.  Support Vector Machines with Applications , 2006, math/0612817.

[2]  Elias Masry,et al.  Multivariate probability density deconvolution for stationary random processes , 1991, IEEE Trans. Inf. Theory.

[3]  M. Hazelton,et al.  Plug-in bandwidth matrices for bivariate kernel density estimation , 2003 .

[4]  Elias Masry,et al.  Deconvolving multivariate kernel density estimates from contaminated associated observations , 2003, IEEE Trans. Inf. Theory.

[5]  Luca Zanni,et al.  Gradient projection methods for quadratic programs and applications in training support vector machines , 2005, Optim. Methods Softw..

[6]  S. Rosset,et al.  Piecewise linear regularized solution paths , 2007, 0708.2197.

[7]  Kurt Hornik,et al.  Support Vector Machines in R , 2006 .

[8]  M. Hazelton,et al.  Comparison of detrital zircon age distributions by kernel functional estimation , 2004 .

[9]  Mark A van de Wiel,et al.  Estimating the False Discovery Rate Using Nonparametric Deconvolution , 2007, Biometrics.

[10]  Density deconvolution based on wavelets with bounded supports , 2002 .

[11]  R. Fletcher.  Practical Methods of Optimization , 1988 .

[12]  Christian H. Hesse,et al.  Data-driven deconvolution , 1999 .

[13]  R. Carroll,et al.  Deconvolving kernel density estimators , 1987 .

[14]  J. Simonoff.  Smoothing Methods in Statistics , 1998 .

[15]  R. Tibshirani,et al.  Least angle regression , 2004, math/0406456.

[16]  Jens Perch Nielsen,et al.  A simple bias reduction method for density estimation , 1995 .

[17]  Gilbert G. Walter.  Density estimation in the presence of noise , 1999 .

[18]  L. Devroye.  Consistent deconvolution in density estimation , 1989 .

[19]  Peihua Qiu,et al.  Discrete-transform approach to deconvolution problems , 2005 .

[20]  Irène Gijbels,et al.  Practical bandwidth selection in deconvolution kernel density estimation , 2004, Comput. Stat. Data Anal..

[21]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[22]  Sam Efromovich,et al.  Density Estimation for the Case of Supersmooth Measurement Error , 1997 .

[23]  Roger Fletcher,et al.  New algorithms for singly linearly constrained quadratic programs subject to lower and upper bounds , 2006, Math. Program..

[24]  Jianqing Fan.  On the Optimal Rates of Convergence for Nonparametric Deconvolution Problems , 1991 .

[25]  Jianqing Fan,et al.  Deconvolution with supersmooth distributions , 1992 .

[26]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[27]  A. Zeileis.  Econometric Computing with HC and HAC Covariance Matrix Estimators , 2004 .

[28]  Ja-Yong Koo,et al.  B-spline deconvolution based on the EM algorithm , 1996 .

[29]  M. R. Osborne,et al.  A new approach to variable selection in least squares problems , 2000 .

[30]  P. Hall,et al.  Optimal Rates of Convergence for Deconvolving a Density , 1988 .

[31]  B. Es,et al.  Asymptotic Normality of Kernel‐Type Deconvolution Estimators , 2005 .

[32]  M. Liu,et al.  Simulations and computations of nonparametric density estimates for the deconvolution problem , 1990 .

[33]  Jianqing Fan,et al.  Global Behavior of Deconvolution Kernel Estimates , 1989 .

[34]  Leonard A. Stefanski,et al.  Rates of convergence of some estimators in a class of deconvolution problems , 1990 .

[35]  S B Hulley,et al.  Overall and coronary heart disease mortality rates in relation to major risk factors in 325,348 men screened for the MRFIT. Multiple Risk Factor Intervention Trial. , 1986, American heart journal.

[36]  Irène Gijbels,et al.  Bootstrap bandwidth selection in kernel density estimation from a contaminated sample , 2004 .

[37]  B. Turlach,et al.  Reducing bias in curve estimation by use of weights , 1999 .

[38]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[39]  B. Vidakovic,et al.  Adaptive wavelet estimator for nonparametric density deconvolution , 1999 .

[40]  Donald Goldfarb,et al.  A numerically stable dual method for solving strictly convex quadratic programs , 1983, Math. Program..

[41]  John Rice,et al.  Deconvolution of Microfluorometric Histograms with B Splines , 1982 .

[42]  Vincent N. LaRiccia,et al.  Nonlinearly Smoothed EM Density Estimation with Automated Smoothing Parameter Selection for Nonparametric Deconvolution Problems , 1997 .

[43]  Kurt Hornik,et al.  kernlab - An S4 Package for Kernel Methods in R , 2004 .

[44]  M. C. Jones,et al.  A reliable data-based bandwidth selection method for kernel density estimation , 1991 .

[45]  D. Ruppert,et al.  Measurement Error in Nonlinear Models , 1995 .

[46]  Philip E. Gill,et al.  Practical optimization , 1981 .

[47]  Roger Fletcher,et al.  Practical methods of optimization; (2nd ed.) , 1987 .

[48]  Raymond J. Carroll,et al.  Measurement error in nonlinear models: a modern perspective , 2006 .

[49]  Matt P. Wand,et al.  Finite sample performance of deconvolving density estimators , 1998 .