Statistical methods with empirical likelihood (EL) are appealing and effective especially in conjunction with estimating equations through which useful data information can be adaptively and flexibly incorporated. It is also known in the literature that EL approaches encounter difficulties when dealing with problems having high-dimensional model parameters and estimating equations. To overcome the challenges, we begin our study with a careful investigation on high-dimensional EL from a new scope targeting at estimating a high-dimensional sparse model parameters. We show that the new scope provides an opportunity for relaxing the stringent requirement on the dimensionality of the model parameter. Motivated by the new scope, we then propose a new penalized EL by applying two penalty functions respectively regularizing the model parameters and the associated Lagrange multipliers in the optimizations of EL. By penalizing the Lagrange multiplier to encourage its sparsity, we show that drastic dimension reduction in the number of estimating equations can be effectively achieved without compromising the validity and consistency of the resulting estimators. Most attractively, such a reduction in dimensionality of estimating equations is actually equivalent to a selection among those high-dimensional estimating equations, resulting in a highly parsimonious and effective device for high-dimensional sparse model parameters. Allowing both the dimensionalities of model parameters and estimating equations growing exponentially with the sample size, our theory demonstrates that the estimator from our new penalized EL is sparse and consistent with asymptotically normally distributed nonzero components. Numerical simulations and a real data analysis show that the proposed penalized EL works promisingly.
[1]
T. Wu,et al.
Nested coordinate descent algorithms for empirical likelihood
,
2014
.
[2]
Cun-Hui Zhang.
Nearly unbiased variable selection under minimax concave penalty
,
2010,
1002.4734.
[3]
M. Treuth,et al.
Multilevel correlates of physical activity for early, mid, and late adolescent girls.
,
2014,
Journal of physical activity & health.
[4]
D. Young,et al.
Predictors for Physical Activity in Adolescent Girls Using Statistical Shrinkage Techniques for Hierarchical Longitudinal Mixed Effects Models
,
2015,
PloS one.
[5]
Peng Zhao,et al.
On Model Selection Consistency of Lasso
,
2006,
J. Mach. Learn. Res..
[6]
Min Tsao,et al.
Empirical likelihood on the full parameter space
,
2013,
1311.1959.
[7]
Chenlei Leng,et al.
Penalized high-dimensional empirical likelihood
,
2010
.
[8]
Zhentao Shi.
Econometric Estimation with High-Dimensional Moment Equalities
,
2016
.
[9]
Min Tsao.
Bounds on coverage probabilities of the empirical likelihood ratio confidence regions
,
2004
.
[10]
K. Lange,et al.
Coordinate descent algorithms for lasso penalized regression
,
2008,
0803.3876.
[11]
Min Tsao,et al.
Extended empirical likelihood for estimating equations
,
2014
.
[12]
Chenlei Leng,et al.
Shrinkage tuning parameter selection with a diverging number of parameters
,
2008
.
[13]
Bing-Yi Jing,et al.
Self-normalized Cramér-type large deviations for independent random variables
,
2003
.
[14]
G. Schwarz.
Estimating the Dimension of a Model
,
1978
.