Adaptive Density Estimation From Data With Small Measurement Errors

In this paper, we study the problem of density estimation from data that contains small measurement errors. The only assumption on these errors is that the maximal measurement error is bounded by some real number converging to zero for sample size tending to infinity. In particular, we do not assume that the measurement errors are independent with expectation zero. We estimate the density by a standard kernel density estimate applied to data with measurement errors and derive a data-driven method to choose its bandwidth. We derive an adaptation result for this estimate and analyze the expected L1 error of our density estimate depending on the smoothness of the density and the size of the maximal measurement error. The results are applied in a density estimation problem in a simulation model, where we show under suitable assumptions that the L1 error of our newly proposed estimate converges to zero much faster than the L1 error of the standard kernel density estimate if both are based on the same number of observations in the simulation model. The performance of the method in case of finite sample size is analyzed using simulated data.

[1]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[2]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[3]  E. Nadaraya.  On Estimating Regression , 1964 .

[4]  G. S. Watson,et al.  Smooth regression analysis , 1964 .

[5]  E. Nadaraya.  Remarks on Non-Parametric Estimates for Density Functions and Regression Curves , 1970 .

[6]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[7]  L. Devroye,et al.  Distribution-Free Consistency Results in Nonparametric Discrimination and Regression Function Estimation , 1980 .

[8]  C. J. Stone,et al.  Optimal Global Rates of Convergence for Nonparametric Regression , 1982 .

[9]  L. Devroye.  Necessary and sufficient conditions for the pointwise convergence of nearest neighbor regression function estimates , 1982 .

[10]  L. Devroye.  The Equivalence of Weak, Strong and Complete Convergence in $L_1$ for Kernel Density Estimates , 1983 .

[11]  L. Devroye.  On arbitrarily slow rates of global convergence in density estimation , 1983 .

[12]  W. Greblicki,et al.  Fourier and Hermite series estimates of regression functions , 1985 .

[13]  L. Devroye,et al.  Nonparametric Density Estimation: The L 1 View. , 1985 .

[14]  Y. Yatracos.  Rates of Convergence of Minimum Distance Estimators and Kolmogorov's Entropy , 1985 .

[15]  L. Devroye.  A Course in Density Estimation , 1987 .

[16]  Ewaryst Rafaj⌈owicz.  Nonparametric orthogonal series estimators of regression: A class attaining the optimal convergence rate in L2☆ , 1987 .

[17]  L. Devroye,et al.  An equivalence theorem for L1 convergence of the kernel regression estimate , 1989 .

[18]  G. Wahba.  Spline models for observational data , 1990 .

[19]  L. Devroye,et al.  No Empirical Probability Measure can Converge in the Total Variation Sense for all Distributions , 1990 .

[20]  G. Lugosi,et al.  On the Strong Universal Consistency of Nearest Neighbor Regression Function Estimates , 1994 .

[21]  Gábor Lugosi,et al.  Nonparametric estimation via empirical risk minimization , 1995, IEEE Trans. Inf. Theory.

[22]  Ewaryst Rafajlowicz,et al.  Consistency of orthogonal series density estimators based on grouped observations , 1997, IEEE Trans. Inf. Theory.

[23]  S. Delattre,et al.  A central limit theorem for normalized functions of the increments of a diffusion process, in the presence of round-off errors , 1997 .

[24]  Axthonv G. Oettinger,et al.  IEEE Transactions on Information Theory , 1998 .

[25]  L. Györfi,et al.  On the asymptotic normality of the L2-error in partitioning regression estimation , 1998 .

[26]  M. Kohler.  Inequalities for uniform deviations of averages from expectations with applications to nonparametric regression , 2000 .

[27]  Adam Krzyzak,et al.  Nonparametric regression estimation using penalized least squares , 2001, IEEE Trans. Inf. Theory.

[28]  Luc Devroye,et al.  Combinatorial methods in density estimation , 2001, Springer series in statistics.

[29]  Ulrich Stadtmüller,et al.  Statistical Aspects of Sampling for Noisy and Grouped Data , 2001 .

[30]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[31]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[32]  A. Meister.  Deconvolution Problems in Nonparametric Statistics , 2009 .

[33]  Schmisser Emeline.  Non-parametric drift estimation for diffusions from noisy data , 2011 .

[34]  F. Comte,et al.  Nonparametric estimation for stochastic differential equations with random effects , 2013 .

[35]  L. Devroye,et al.  Estimation of a distribution from data with small measurement errors , 2013 .

[36]  Michael Kohler,et al.  Optimal global rates of convergence for noiseless regression estimation problems with adaptively chosen design , 2014, J. Multivar. Anal..

[37]  A. Krzyżak,et al.  Adaptive density estimation based on real and artificial data , 2015 .