Smoothed Nonparametric Derivative Estimation using Weighted Difference Quotients

Derivatives play an important role in bandwidth selection methods (e.g., plug-ins), data analysis and bias-corrected confidence intervals. Therefore, obtaining accurate derivative information is crucial. Although many derivative estimation methods exist, the majority require a fixed design assumption. In this paper, we propose an effective and fully datadriven framework to estimate the first and second order derivative in random design. We establish the asymptotic properties of the proposed derivative estimator, and also propose a fast selection method for the tuning parameters. The performance and flexibility of the method is illustrated via an extensive simulation study.

[1]  Wenwu Wang,et al.  Derivative estimation based on difference sequence via locally weighted least squares regression , 2015, J. Mach. Learn. Res..

[2]  J. Opsomer,et al.  Plug-in bandwidth selector for local polynomial regression estimator with correlated errors , 2004 .

[3]  Irène Gijbels,et al.  Data-Driven Discontinuity Detection in Derivatives of a Regression Function , 2005 .

[4]  James O. Ramsay,et al.  Applied Functional Data Analysis: Methods and Case Studies , 2002 .

[5]  J. Opsomer,et al.  Local polynomial regression with correlated errors in random design and unknown correlation structure , 2018, Biometrika.

[6]  Jane-Ling Wang.  Nonparametric Regression Analysis of Longitudinal Data , 2005 .

[7]  Babak Hassibi,et al.  Second Order Derivatives for Network Pruning: Optimal Brain Surgeon , 1992, NIPS.

[8]  P. Hall,et al.  Asymptotically optimal difference-based estimation of variance in nonparametric regression , 1990 .

[9]  D. Wolfe,et al.  ON DERIVATIVE ESTIMATION IN SPLINE REGRESSION , 2000 .

[10]  Cidambi Srinivasan,et al.  A Generalized Cp Criterion for Derivative Estimation , 2011, Technometrics.

[11]  Ulrich Stadtmüller,et al.  Bandwidth choice and confidence intervals for derivatives of noisy data , 1987 .

[12]  Yu Liu,et al.  Derivative Estimation in Random Design , 2018, NeurIPS.

[13]  Bart De Moor,et al.  Derivative estimation with local polynomial fitting , 2013, J. Mach. Learn. Res..

[14]  Robert Haining,et al.  Statistics for spatial data: by Noel Cressie, 1991, John Wiley & Sons, New York, 900 p., ISBN 0-471-84336-9, US $89.95 , 1993 .

[15]  W. Härdle.  Applied Nonparametric Regression , 1992 .

[16]  C. J. Stone,et al.  Additive Regression and Other Nonparametric Models , 1985 .

[17]  E. Parzen.  On Estimation of a Probability Density Function and Mode , 1962 .

[18]  M. Delecroix,et al.  Nonparametric estimation of a regression function and its derivatives under an ergodic hypothesis , 1996 .

[19]  Yingcun Xia,et al.  Bias‐corrected confidence bands in nonparametric regression , 1998 .

[20]  Matthew P. Wand,et al.  Kernel Smoothing , 1995 .

[21]  A. Meister.  Deconvolution Problems in Nonparametric Statistics , 2009 .

[22]  J. S. Marron,et al.  SiZer for time series: A new approach to the analysis of trends , 2007, 0706.4190.

[23]  G. Wahba,et al.  When is the optimal regularization parameter insensitive to the choice of the loss function , 1990 .

[24]  Xin Fu,et al.  Confidence bands in nonparametric regression , 2009 .

[25]  Adam Krzyzak,et al.  A Distribution-Free Theory of Nonparametric Regression , 2002, Springer series in statistics.

[26]  Marc G. Genton,et al.  Optimal Estimation of Derivatives in Nonparametric Regression , 2016, J. Mach. Learn. Res..

[27]  J. Marron,et al.  SiZer for Exploration of Structures in Curves , 1999 .

[28]  M. Rosenblatt.  Remarks on Some Nonparametric Estimates of a Density Function , 1956 .

[29]  Jianqing Fan.  Local Polynomial Modelling and Its Applications: Monographs on Statistics and Applied Probability 66 , 1996 .

[30]  G. Akrivis.  A First Course In The Numerical Analysis Of Differential Equations [Book News & Reviews] , 1998, IEEE Computational Science and Engineering.

[31]  Alexandre B. Tsybakov,et al.  Introduction to Nonparametric Estimation , 2008, Springer series in statistics.

[32]  J. Rice.  Bandwidth choice for differentiation , 1986 .

[33]  Yuhong Yang,et al.  Nonparametric Regression with Correlated Errors , 2001 .

[34]  Cheolwoo Park,et al.  SiZer analysis for the comparison of regression curves , 2008, Comput. Stat. Data Anal..