Penalized least squares for single index models

The single index model is a useful regression model. In this paper, we propose a nonconcave penalized least squares method to estimate both the parameters and the link function of the single index model. Compared to other variable selection and estimation methods, the proposed method can estimate parameters and select variables simultaneously. When the dimension of parameters in the single index model is a fixed constant, under some regularity conditions, we demonstrate that the proposed estimators for parameters have the so-called oracle property, and furthermore we establish the asymptotic normality and develop a sandwich formula to estimate the standard deviations of the proposed estimators. Simulation studies and a real data analysis are presented to illustrate the proposed methods.

[1]  A. Juditsky,et al.  Direct estimation of the index coefficient in a single-index model , 2001 .

[2]  Runze Li,et al.  Tuning parameter selectors for the smoothly clipped absolute deviation method. , 2007, Biometrika.

[3]  Jianqing Fan,et al.  Generalized Partially Linear Single-Index Models , 1997 .

[4]  M. Wand,et al.  An Effective Bandwidth Selector for Local Least Squares Regression , 1995 .

[5]  Yingcun Xia,et al.  Variable selection for the single‐index model , 2007 .

[6]  H. Zou.  The Adaptive Lasso and Its Oracle Properties , 2006 .

[7]  W. D. McArdle,et al.  Nutrition, weight control, and exercise , 1977 .

[8]  Jianqing Fan,et al.  Variable Selection for Cox's proportional Hazards Model and Frailty Model , 2002 .

[9]  Thomas M. Stoker,et al.  Semiparametric Estimation of Index Coefficients , 1989 .

[10]  A. K. Solomon,et al.  Advances in Biological and Medical Physics , 1949 .

[11]  D. Hunter,et al.  Variable Selection using MM Algorithms. , 2005, Annals of statistics.

[12]  Jianqing Fan,et al.  Local polynomial modelling and its applications , 1994 .

[13]  W. Härdle,et al.  Direct Semiparametric Estimation of Single-Index Models with Discrete Covariates dpsfb950075.ps.tar = Enno MAMMEN J.S. MARRON: Mass Recentered Kernel Smoothers , 1996 .

[14]  Jianqing Fan,et al.  Nonconcave penalized likelihood with a diverging number of parameters , 2004, math/0406466.

[15]  Runze Li,et al.  Variable Selection in Semiparametric Regression Modeling. , 2008, Annals of statistics.

[16]  Jianqing Fan,et al.  Variable Selection via Nonconcave Penalized Likelihood and its Oracle Properties , 2001 .

[17]  S. Geer.  Empirical Processes in M-Estimation , 2000 .

[18]  S. Weisberg,et al.  Comments on "Sliced inverse regression for dimension reduction" by K. C. Li , 1991 .

[19]  A. G. Fisher,et al.  Generalized body composition prediction equations for men using simple measurement techniques , 1985 .

[20]  Yuehua Wu,et al.  Model selection with data-oriented penalty , 1999 .

[21]  高等学校計算数学学報編輯委員会編,et al.  高等学校計算数学学報 = Numerical mathematics , 1979 .

[22]  A. Tsybakov,et al.  Sliced Inverse Regression for Dimension Reduction - Comment , 1991 .

[23]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[24]  Jianqing Fan,et al.  Comments on «Wavelets in statistics: A review» by A. Antoniadis , 1997 .

[25]  Jianqing Fan.  Nonlinear Time Series , 2003 .

[26]  Ker-Chau Li,et al.  Slicing Regression: A Link-Free Regression Method , 1991 .

[27]  W. Härdle,et al.  Optimal Smoothing in Single-index Models , 1993 .

[28]  Jianqing Fan,et al.  COMMENTS ON « WAVELETS IN STATISTICS : A REVIEW , 2009 .

[29]  A. Tsybakov,et al.  How sensitive are average derivatives , 1993 .

[30]  W. Siri,et al.  The gross composition of the body. , 1956, Advances in biological and medical physics.

[31]  H. Zou,et al.  One-step Sparse Estimates in Nonconcave Penalized Likelihood Models. , 2008, Annals of statistics.

[32]  H. Ichimura,et al.  SEMIPARAMETRIC LEAST SQUARES (SLS) AND WEIGHTED SLS ESTIMATION OF SINGLE-INDEX MODELS , 1993 .

[33]  Prasad A. Naik,et al.  Single‐index model selections , 2001 .

[34]  Thomas M. Stoker.  Consistent estimation of scaled coefficients , 2011 .

[35]  Li-Xing Zhu,et al.  Nonconcave penalized inverse regression in single-index models with high dimensional predictors , 2009, J. Multivar. Anal..