Wavelet regression and additive models for irregularly spaced data

We present a novel approach for nonparametric regression using wavelet basis functions. Our proposal, waveMesh, can be applied to non-equispaced data with sample size not necessarily a power of 2. We develop an efficient proximal gradient descent algorithm for computing the estimator and establish adaptive minimax convergence rates. The main appeal of our approach is that it naturally extends to additive and sparse additive models for a potentially large number of covariates. We prove minimax optimal convergence rates under a weak compatibility condition for sparse additive models. The compatibility condition holds when we have a small number of covariates. Additionally, we establish convergence rates for when the condition is not met. We complement our theoretical results with empirical studies comparing waveMesh to existing methods.

[1]  J. Lafferty,et al.  Sparse additive models , 2007, 0711.4555.

[2]  Michael R. Chernick,et al.  Wavelet Methods for Time Series Analysis , 2001, Technometrics.

[3]  Marina I. Knight,et al.  adlift: An Adaptive Lifting Scheme Algorithm , 2005 .

[4]  Sujit K. Ghosh,et al.  Essential Wavelets for Statistical Applications and Data Analysis , 2001, Technometrics.

[5]  Y. Nesterov Gradient methods for minimizing composite objective function , 2007 .

[6]  I. Daubechies Orthonormal bases of compactly supported wavelets , 1988 .

[7]  S. Geer Empirical Processes in M-Estimation , 2000 .

[8]  M.E.Sc. Wieslaw Stepniewski,et al.  The Prediction of Performance , 2013 .

[9]  Anestis Antoniadis,et al.  Adaptive wavelet series estimation in separable nonparametric regression models , 2001, Stat. Comput..

[10]  B. Silverman,et al.  Some Aspects of the Spline Smoothing Approach to Non‐Parametric Regression Curve Fitting , 1985 .

[11]  I. Johnstone,et al.  Ideal spatial adaptation by wavelet shrinkage , 1994 .

[12]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Y. Meyer,et al.  Wavelets and Filter Banks , 1991 .

[14]  Jianqing Fan,et al.  Regularization of Wavelet Approximations , 2001 .

[15]  Brani Vidakovic,et al.  On Non-Equally Spaced Wavelet Regression , 2001 .

[16]  Arne Kovac,et al.  Extending the Scope of Wavelet Regression Methods by Coefficient-Dependent Thresholding , 2000 .

[17]  Amara Lynn Graps,et al.  An introduction to wavelets , 1995 .

[18]  I. Johnstone,et al.  Adapting to Unknown Smoothness via Wavelet Shrinkage , 1995 .

[19]  S. Geer,et al.  On the conditions used to prove oracle results for the Lasso , 2009, 0910.0722.

[20]  Sara van de Geer,et al.  Estimation and Testing Under Sparsity: École d'Été de Probabilités de Saint-Flour XLV – 2015 , 2016 .

[21]  Ashley Petersen,et al.  Fused Lasso Additive Model , 2014, Journal of computational and graphical statistics : a joint publication of American Statistical Association, Institute of Mathematical Statistics, Interface Foundation of North America.

[22]  A. Dalalyan,et al.  On the Prediction Performance of the Lasso , 2014, 1402.1700.

[23]  I. Daubechies Orthonormal bases of compactly supported wavelets II: variations on a theme , 1993 .

[24]  Empirical Wavelet-based Estimation for Non-linear Additive Regression Models , 2018, 1803.04558.

[25]  David L. Donoho,et al.  De-noising by soft-thresholding , 1995, IEEE Trans. Inf. Theory.

[26]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[27]  P. Tseng,et al.  AMlet, RAMlet, and GAMlet: Automatic Nonlinear Fitting of Additive Models, Robust and Generalized, With Wavelets , 2004 .

[28]  G. Wahba Spline models for observational data , 1990 .

[29]  Shuanglin Zhang,et al.  Wavelet threshold estimation for additive regression models , 2003 .

[30]  R. Tibshirani The Lasso Problem and Uniqueness , 2012, 1206.0313.

[31]  B. Turlach,et al.  Interpolation methods for nonlinear wavelet regression with irregularly spaced design , 1997 .

[32]  Ingrid Daubechies,et al.  The wavelet transform, time-frequency localization and signal analysis , 1990, IEEE Trans. Inf. Theory.

[33]  Donald B. Percival,et al.  Wavelet shrinkage for unequally spaced data , 1999, Stat. Comput..

[34]  T. Tony Cai,et al.  Wavelet estimation for samples with random uniform design , 1999 .

[35]  Guy P. Nason,et al.  Adaptive lifting for nonparametric regression , 2006, Stat. Comput..

[36]  Y. Meyer Principe d'incertitude, bases hilbertiennes et algèbres d'opérateurs , 1986 .

[37]  T. Tony Cai,et al.  WAVELET SHRINKAGE FOR NONEQUISPACED SAMPLES , 1998 .

[38]  Guy P. Nason,et al.  Wavelet Methods in Statistics with R , 2008 .