Nonparametric Estimation of Regression Functions in the Presence of Irrelevant Regressors

In this paper we consider a nonparametric regression model that admits a mix of continuous and discrete regressors, some of which may in fact be redundant (that is, irrelevant). We show that, asymptotically, a data-driven least squares cross-validation method can remove irrelevant regressors. Simulations reveal that this automatic dimensionality reduction feature is very effective in finite-sample settings.

[1]  R. M. Clark.  A calibration curve for radiocarbon dates , 1975, Antiquity.

[2]  J. Aitchison,et al.  Multivariate binary discrimination by the kernel method , 1976 .

[3]  P. Hall.  On nonparametric multivariate binary discrimination , 1981 .

[4]  W. Härdle,et al.  Optimal Bandwidth Selection in Nonparametric Regression Function Estimation , 1985 .

[5]  M. Wand,et al.  On nonparametric discrimination using density differences , 1988 .

[6]  W. Härdle,et al.  How Far are Automatically Chosen Regression Smoothing Parameters from their Optimum , 1988 .

[7]  Gerhard Tutz,et al.  Sequential models in categorical regression , 1991 .

[8]  James Stephen Marron,et al.  Regression smoothing parameters that are not far from their optimum , 1992 .

[9]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[10]  P. Hall,et al.  On the performance of kernel estimators for high-dimensional, sparse binary data , 1993 .

[11]  Jianqing Fan,et al.  Data‐Driven Bandwidth Selection in Local Polynomial Fitting: Variable Bandwidth and Spatial Adaptation , 1995 .

[12]  田中 豊.  Multivariate Statistical Modelling Based on Generalized Linear Models/Ludwig Fahrmeir,Gerhard Tutz(1994) , 1995 .

[13]  Yanqin Fan,et al.  Consistent model specification tests : Omitted variables and semiparametric functional forms , 1996 .

[14]  Quang Vuong,et al.  Nonparametric Selection of Regressors: The Nonnested Case , 1996 .

[15]  Bruce Western,et al.  Vague Theory and Model Uncertainty in Macrosociology , 1996 .

[16]  Jeffrey S. Racine,et al.  Consistent Significance Testing for Nonparametric Regression , 1997 .

[17]  J. Hahn.  On the Role of the Propensity Score in Efficient Semiparametric Estimation of Average Treatment Effects , 1998 .

[18]  J. Simonoff.  Smoothing Methods in Statistics , 1998 .

[19]  G. Imbens,et al.  Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score , 2000 .

[20]  Quang Vuong,et al.  NONPARAMETRIC SIGNIFICANCE TESTING , 2000, Econometric Theory.

[21]  Jeffrey S. Racine,et al.  Nonparametric estimation of regression functions with both categorical and continuous data , 2004 .

[22]  Jeffrey S. Racine,et al.  Cross-Validation and the Estimation of Conditional Probability Densities , 2004 .

[23]  Qi Li,et al.  Nonparametric Econometrics: Theory and Practice , 2006 .