On the Validity of the Pairs Bootstrap for Lasso Estimators

We study the validity of the pairs bootstrap for lasso estimators in linear regression models with random covariates and heteroscedastic error terms. We show that the naive pairs bootstrap does not provide a valid method for approximating the distribution of the lasso estimator. To overcome this deficiency, we introduce a modified pairs bootstrap procedure and prove its consistency. Finally, we consider the adaptive lasso and show that the modified pairs bootstrap consistently estimates the distribution of the adaptive lasso estimator.

[1]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[2]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[3]  S. Lahiri,et al.  Bootstrapping Lasso Estimators , 2011 .

[4]  F. Audrino,et al.  Oracle Properties and Finite Sample Inference of the Adaptive Lasso for Time Series Regression Models , 2013, 1312.1473.

[5]  C. Geyer.  On the Asymptotics of Constrained $M$-Estimation , 1994 .

[6]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[7]  Sara van de Geer,et al.  Statistics for High-Dimensional Data , 2011 .

[8]  Peter Hall,et al.  BOOTSTRAP-BASED PENALTY CHOICE FOR THE LASSO , ACHIEVING ORACLE PERFORMANCE , 2009 .

[9]  Sara van de Geer,et al.  Statistics for High-Dimensional Data: Methods, Theory and Applications , 2011 .

[10]  Martin J. Wainwright,et al.  Sharp thresholds for high-dimensional and noisy recovery of sparsity , 2006, ArXiv.

[11]  Lu Tian,et al.  A Perturbation Method for Inference on Regularized Regression Estimates , 2011, Journal of the American Statistical Association.

[12]  S. Geer,et al.  On asymptotically optimal confidence regions and tests for high-dimensional models , 2013, 1303.0518.

[13]  Wenjiang J. Fu,et al.  Asymptotics for lasso-type estimators , 2000 .

[14]  Holger Dette,et al.  Bridge estimators and the adaptive lasso under heteroscedasticity , 2012 .

[15]  S. Lahiri,et al.  Rates of convergence of the Adaptive LASSO estimators to the Oracle distribution and higher order refinements by the bootstrap , 2013, 1307.1952.

[16]  J. Fox.  Bootstrapping Regression Models , 2002 .

[17]  R. Tibshirani,et al.  A SIGNIFICANCE TEST FOR THE LASSO. , 2013, Annals of statistics.

[18]  S. N. Lahiri,et al.  Asymptotic properties of the residual bootstrap for Lasso estimators , 2010 .

[19]  R. Tibshirani,et al.  Regression shrinkage and selection via the lasso: a retrospective , 2011 .

[20]  Cun-Hui Zhang,et al.  Confidence intervals for low dimensional parameters in high dimensional linear models , 2011, 1110.2563.