Partial Consistency with Sparse Incidental Parameters.

Penalized estimation principle is fundamental to high-dimensional problems. In the literature, it has been extensively and successfully applied to various models with only structural parameters. As a contrast, in this paper, we first apply this penalization principle to a linear regression model with a finite-dimensional vector of structural parameters and a high-dimensional vector of sparse incidental parameters. For the estimators of the structural parameters, we derive their consistency and asymptotic normality, which reveals an oracle property. However, the penalized estimators for the incidental parameters possess only partial selection consistency but not consistency. This is an interesting partial consistency phenomenon: the structural parameters are consistently estimated while the incidental ones cannot. For the structural parameters, also considered is an alternative two-step penalized estimator, which has fewer possible asymptotic distributions and thus is more suitable for statistical inferences. We further extend the methods and results to the case where the dimension of the structural parameter vector diverges with but slower than the sample size. A data-driven approach for selecting a penalty regularization parameter is provided. The finite-sample performance of the penalized estimators for the structural parameters is evaluated by simulations and a real data set is analyzed.

[1]  M. Kosorok,et al.  Likelihood based inference for current status data on a grid: A boundary phenomenon and an adaptive inference procedure , 2012, 1205.6055.

[2]  J. Jahn.  Introduction to the Theory of Nonlinear Optimization , 1994 .

[3]  Martin Raič,et al.  Normal Approximation by Stein ’ s Method , 2003 .

[4]  M. Kosorok.  Introduction to Empirical Processes and Semiparametric Inference , 2008 .

[5]  Jianqing Fan,et al.  A Selective Overview of Variable Selection in High Dimensional Feature Space. , 2009, Statistica Sinica.

[6]  J. Neyman,et al.  Consistent Estimates Based on Partially Consistent Observations , 1948 .

[7]  Yang Feng,et al.  A road to classification in high dimensional space: the regularized optimal affine discriminant , 2010, Journal of the Royal Statistical Society. Series B, Statistical methodology.

[8]  Laurent Zwald,et al.  Robust regression through the Huber’s criterion and adaptive lasso penalty , 2011 .

[9]  Marcelo J. Moreira.  A Maximum Likelihood Method for the Incidental Parameter Problem , 2008, 0909.0613.

[10]  Elizabeth L. Scott,et al.  Consistent Estimates Based on Partially Consistent Observations Author ( s ) : , 2007 .

[11]  T. Lancaster.  The incidental parameter problem since 1948 , 2000 .

[12]  Frederick R. Forst,et al.  On robust estimation of the location parameter , 1980 .

[13]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[14]  Thomas P. Wihler,et al.  On the Hölder Continuity of Matrix Functions for Normal Matrices , 2009 .

[15]  Noureddine El Karoui,et al.  Penalized robust regression in high-dimension , 2011 .

[16]  J. Kiefer,et al.  CONSISTENCY OF THE MAXIMUM LIKELIHOOD ESTIMATOR IN THE PRESENCE OF INFINITELY MANY INCIDENTAL PARAMETERS , 1956 .

[17]  Jianqing Fan,et al.  Nonconcave penalized likelihood with a diverging number of parameters , 2004, math/0406466.

[18]  Peng Zhao,et al.  On Model Selection Consistency of Lasso , 2006, J. Mach. Learn. Res..

[19]  J. Ghosh.  Elimination of Nuisance Parameters , 1988 .

[20]  Jianqing Fan,et al.  High Dimensional Covariance Matrix Estimation in Approximate Factor Models , 2011, Annals of statistics.

[21]  Stephen Portnoy,et al.  A Robust Journey in the New Millennium , 2000 .

[22]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[23]  Jianqing Fan,et al.  ADAPTIVE ROBUST VARIABLE SELECTION. , 2012, Annals of statistics.

[24]  G. Stewart.  On the Continuity of the Generalized Inverse , 1969 .

[25]  Clifford Lam.  High‐dimensional covariance matrix estimation , 2019, WIREs Computational Statistics.

[26]  P. J. Huber.  Robust Regression: Asymptotics, Conjectures and Monte Carlo , 1973 .