Density estimation using real and artificial data

Let X, X<sub>1</sub>, X<sub>2</sub>, ... be independent and identically distributed ℝ<sup>d</sup>-valued random variables and let m : ℝ<sup>d</sup> → ℝ be an unknown measurable function such that a density f of Y = m(X) exists. In this paper we consider estimating f based on i.i.d. sample (X<sub>1</sub>, Y<sub>1</sub>);...; (X<sub>n</sub>, Y<sub>n</sub>) of (X, Y) and on additional independent observations of X. We compare the standard kernel density estimate based on the y-values of the sample of (X, Y) and a kernel density estimate based on artificially generated y-values corresponding to the additional observations of X. It is shown that under suitable smoothness assumptions on f and m the rate of convergence of the L<sub>1</sub> error of the latter estimate is better than that of the standard kernel density estimate. Furthermore, a density estimate defined as convex combination of these two estimates is considered and a data-driven choice of the bandwidths and the weight of the convex combination is proposed and investigated.

[1]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[2]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[3]  E. Nadaraya.  On Estimating Regression , 1964 .

[4]  G. S. Watson,et al.  Smooth regression analysis , 1964 .

[5]  E. Nadaraya.  Remarks on Non-Parametric Estimates for Density Functions and Regression Curves , 1970 .

[6]  C. J. Stone,et al.  Consistent Nonparametric Regression , 1977 .

[7]  L. Devroye,et al.  Distribution-Free Consistency Results in Nonparametric Discrimination and Regression Function Estimation , 1980 .

[8]  C. J. Stone,et al.  Optimal Global Rates of Convergence for Nonparametric Regression , 1982 .

[9]  L. Devroye.  Necessary and sufficient conditions for the pointwise convergence of nearest neighbor regression function estimates , 1982 .

[10]  L. Devroye.  The Equivalence of Weak, Strong and Complete Convergence in $L_1$ for Kernel Density Estimates , 1983 .

[11]  L. Devroye.  A Course in Density Estimation , 1987 .

[12]  L. Devroye,et al.  An equivalence theorem for L1 convergence of the kernel regression estimate , 1989 .

[13]  G. Wahba.  Spline models for observational data , 1990 .

[14]  Grace Wahba,et al.  8. Fredholm Integral Equations of the First Kind , 1990 .

[15]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[16]  G. Lugosi,et al.  On the Strong Universal Consistency of Nearest Neighbor Regression Function Estimates , 1994 .

[17]  Gábor Lugosi,et al.  Nonparametric estimation via empirical risk minimization , 1995, IEEE Trans. Inf. Theory.

[18]  Matthew P. Wand,et al.  Kernel Smoothing , 1995 .

[19]  L. Györfi,et al.  On the asymptotic normality of the L2-error in partitioning regression estimation , 1998 .

[20]  M. Kohler.  Inequalities for uniform deviations of averages from expectations with applications to nonparametric regression , 2000 .

[21]  Adam Krzyzak,et al.  Nonparametric regression estimation using penalized least squares , 2001, IEEE Trans. Inf. Theory.

[22]  Luc Devroye,et al.  Combinatorial methods in density estimation , 2001, Springer series in statistics.

[23]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[24]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[25]  P. Larsen.  Maximum Penalized Likelihood Estimation, Volume I, Density Estimation , 2004 .

[26]  B. Silverman,et al.  Maximum Penalized Likelihood Estimation , 2006 .

[27]  Daniel Stefankovic,et al.  Density Estimation in Linear Time , 2007, COLT.

[28]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[29]  Amaury Lendasse,et al.  Residual variance estimation using a nearest neighbor statistic , 2010, J. Multivar. Anal..

[30]  Ricardo Cao,et al.  Nonparametric Density Estimation , 2013 .

[31]  Luc Devroye,et al.  Estimation of a Density Using Real and Artificial Data , 2013, IEEE Transactions on Information Theory.

[32]  A. Krzyżak,et al.  Optimal global rates of convergence for interpolation problems with random design , 2013 .

[33]  Gou Hosoya,et al.  国際会議参加報告:2014 IEEE International Symposium on Information Theory , 2014 .

[34]  A. Krzyżak,et al.  Adaptive density estimation based on real and artificial data , 2015 .