Asymptotically Honest Confidence Regions for High Dimensional Parameters by the Desparsified Conservative Lasso

In this paper we consider the conservative Lasso which we argue penalizes more correctly than the Lasso and show how it may be desparsified in the sense of van de Geer et al. (2014) in order to construct asymptotically honest (uniform) confidence bands. In particular, we develop an oracle inequality for the conservative Lasso only assuming the existence of a certain number of moments. This is done by means of the Marcinkiewicz-Zygmund inequality. We allow for heteroskedastic non-subgaussian error terms and covariates. Next, we desparsify the conservative Lasso estimator and derive the asymptotic distribution of tests involving an increasing number of parameters. Our simulations reveal that the desparsified conservative Lasso estimates the parameters more precisely than the desparsified Lasso, has better size properties and produces confidence bands with superior coverage rates.

[1]  Adel Javanmard,et al.  Hypothesis Testing in High-Dimensional Regression Under the Gaussian Random Design Model: Asymptotic Theory , 2013, IEEE Transactions on Information Theory.

[2]  R. Tibshirani,et al.  Regression shrinkage and selection via the lasso: a retrospective , 2011 .

[3]  S. Geer,et al.  On asymptotically optimal confidence regions and tests for high-dimensional models , 2013, 1303.0518.

[4]  L. J. Savage,et al.  The nonexistence of certain statistical procedures in nonparametric problems , 1956 .

[5]  Paolo Santucci de Magistris,et al.  Chasing Volatility: A Persistent Multiplicative Error Model with Jumps , 2014 .

[6]  M. Nielsen,et al.  A Fractionally Cointegrated VAR Analysis of Economic Voting and Political Support , 2014 .

[7]  A. Belloni,et al.  Inference for High-Dimensional Sparse Econometric Models , 2011, 1201.0220.

[8]  S. Geer,et al.  On the conditions used to prove oracle results for the Lasso , 2009, 0910.0722.

[9]  Victor Chernozhukov,et al.  High Dimensional Sparse Econometric Models: An Introduction , 2011, 1106.5242.

[10]  H. Zou,et al.  One-step Sparse Estimates in Nonconcave Penalized Likelihood Models. , 2008, Annals of statistics.

[11]  Asger Lunde,et al.  Factor Structure in Commodity Futures Return and Volatility , 2017, Journal of Financial and Quantitative Analysis.

[12]  Cun-Hui Zhang,et al.  Confidence intervals for low dimensional parameters in high dimensional linear models , 2011, 1110.2563.

[13]  Jianqing Fan,et al.  Power Enhancement in High Dimensional Cross-Sectional Tests , 2013, Econometrica : journal of the Econometric Society.

[14]  Ker-Chau Li,et al.  Honest Confidence Regions for Nonparametric Regression , 1989 .

[15]  A. Belloni,et al.  Least Squares After Model Selection in High-Dimensional Sparse Models , 2009, 1001.0188.

[16]  Dennis Kristensen,et al.  ABC of SV: Limited Information Likelihood Inference in Stochastic Volatility Jump-Diffusion Models , 2015 .

[17]  Adel Javanmard,et al.  Confidence intervals and hypothesis testing for high-dimensional regression , 2013, J. Mach. Learn. Res..

[18]  Anders Bredahl Kock,et al.  ORACLE EFFICIENT VARIABLE SELECTION IN RANDOM AND FIXED EFFECTS PANEL DATA MODELS , 2013, Econometric Theory.

[19]  Karim Lounici.  Sup-norm convergence rate and sign concentration property of Lasso and Dantzig estimators , 2008, 0801.4610.

[20]  A. Belloni,et al.  SPARSE MODELS AND METHODS FOR OPTIMAL INSTRUMENTS WITH AN APPLICATION TO EMINENT DOMAIN , 2012 .

[21]  J. Horowitz,et al.  Asymptotic properties of bridge estimators in sparse high-dimensional regression models , 2008, 0804.0693.

[22]  Ke-Li Xu,et al.  A Fractionally Cointegrated VAR Analysis of Price Discovery in Commodity Futures Markets , 2015 .

[23]  Sara van de Geer,et al.  Statistical Theory for High-Dimensional Models , 2014, 1409.8557.

[24]  Peter Bühlmann.  Regression shrinkage and selection via the Lasso: a retrospective (Robert Tibshirani): Comments on the presentation , 2011 .

[25]  Benedikt M. Pötscher.  Confidence Sets Based on Sparse Estimators Are Necessarily Large , 2007 .

[26]  H. Zou,et al.  STRONG ORACLE OPTIMALITY OF FOLDED CONCAVE PENALIZED ESTIMATION. , 2012, Annals of statistics.

[27]  Jianqing Fan,et al.  Sure independence screening for ultrahigh dimensional feature space , 2006, math/0612857.

[28]  Mikko S. Pakkanen,et al.  Discretization of L\'evy semistationary processes with application to estimation , 2014, 1407.2754.

[29]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[30]  Kengo Kato,et al.  Uniform post selection inference for LAD regression and other z-estimation problems , 2013 .

[31]  Jianqing Fan,et al.  ADAPTIVE ROBUST VARIABLE SELECTION. , 2012, Annals of statistics.

[32]  M. Nielsen,et al.  Bootstrap Score Tests for Fractional Integration in Heteroskedastic ARFIMA Models, with an Application to Price Dynamics in Commodity Spot and Futures Markets , 2013 .

[33]  Tommaso Proietti,et al.  Outlier Detection in Structural Time Series Models: The Indicator Saturation Approach , 2014 .

[34]  Yingying Fan,et al.  Tuning parameter selection in high dimensional penalized likelihood , 2013, 1605.03321.

[35]  Jonathan Taylor,et al.  Statistical learning and selective inference , 2015, Proceedings of the National Academy of Sciences.

[36]  A. Belloni,et al.  Inference on Treatment Effects after Selection Amongst High-Dimensional Controls , 2011, 1201.0224.

[37]  R. Tibshirani,et al.  A SIGNIFICANCE TEST FOR THE LASSO. , 2013, Annals of statistics.

[38]  M. D. Cattaneo,et al.  Bootstrapping Kernel-Based Semiparametric Estimators , 2014 .

[39]  P. Bickel,et al.  SIMULTANEOUS ANALYSIS OF LASSO AND DANTZIG SELECTOR , 2008, 0801.1095.

[40]  Mehmet Caner,et al.  Adaptive Elastic Net for Generalized Methods of Moments , 2014, Journal of business & economic statistics : a publication of the American Statistical Association.

[41]  N. Haldrup,et al.  Discriminating between fractional integration and spurious long memory , 2014 .

[42]  B. M. Pötscher,et al.  MODEL SELECTION AND INFERENCE: FACTS AND FICTION , 2005, Econometric Theory.

[43]  W Y Zhang,et al.  Discussion on `Sure independence screening for ultra-high dimensional feature space' by Fan, J and Lv, J. , 2008 .

[44]  P. Robinson.  ROOT-N-CONSISTENT SEMIPARAMETRIC REGRESSION , 1988 .

[45]  Markku Lanne,et al.  Is the Quantity Theory of Money Useful in Forecasting U.S. Inflation , 2014 .

[46]  A. Buja,et al.  Valid post-selection inference , 2013, 1306.1059.

[47]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[48]  Marc Hoffmann,et al.  On adaptive inference and confidence bands , 2011, 1202.5145.

[49]  Victor Chernozhukov,et al.  Pivotal estimation via square-root Lasso in nonparametric regression , 2014 .

[50]  Jianqing Fan,et al.  A Selective Overview of Variable Selection in High Dimensional Feature Space. , 2009, Statistica Sinica.

[51]  N. Meinshausen,et al.  High-dimensional graphs and variable selection with the Lasso , 2006, math/0608017.

[52]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[53]  Sara van de Geer,et al.  Statistics for High-Dimensional Data , 2011 .

[54]  Sara van de Geer,et al.  Confidence sets in sparse regression , 2012, 1209.1508.

[55]  Ming Yuan,et al.  High Dimensional Inverse Covariance Matrix Estimation via Linear Programming , 2010, J. Mach. Learn. Res..