A new approach to the variable selection using the TLS approach

The problem of variable selection is one of the most pervasive model selection problems in statistical applications. It arises when one wants to explain the observations or data adequately by a subset of possible regression variables. The objective is to identify factors of importance and to include only variables that contribute significantly to the reduction of the prediction error. Numerous selection procedures have been proposed in the classical multiple linear regression model. We extend one of the most popular methods developed in this context, the backward selection procedure, to a more general class of models. In the basic linear regression model, errors are present on the observations only, if errors are present on the regressors as well, one gets the errors-in-variables model which for Gaussian noise becomes the total-least-squares model, this is the context considered here.

[1]  Dean P. Foster,et al.  Calibration and empirical Bayes variable selection , 2000 .

[2]  James Durbin,et al.  Errors in variables , 1954 .

[3]  N. Draper,et al.  Applied Regression Analysis , 1966 .

[4]  C. H. Oh,et al.  Some comments on , 1998 .

[5]  H. Luetkepohl The Handbook of Matrices , 1996 .

[6]  L. Gleser Estimation in a Multivariate "Errors in Variables" Regression Model: Large Sample Results , 1981 .

[7]  Toshiro Haga,et al.  Selection of variables in multiple regression analysis , 1976 .

[8]  G. Stewart,et al.  Matrix Perturbation Theory , 1990 .

[9]  Ahmed H. Tewfik,et al.  Deterministic and iterative solutions to subset selection problems , 2002, IEEE Trans. Signal Process..

[10]  H. Akaike A new look at the statistical model identification , 1974 .

[11]  Leland Wilkinson,et al.  Tests of Significance in Forward Selection Regression With an F-to-Enter Stopping Rule , 1981 .

[12]  G. Williamson,et al.  QR-based TLS and mixed LS-TLS algorithms with applications to adaptive IIR filtering , 2003, IEEE Trans. Signal Process..

[13]  George A. F. Seber,et al.  Linear regression analysis , 1977 .

[14]  R. Tibshirani Regression Shrinkage and Selection via the Lasso , 1996 .

[15]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[16]  Sabine Van Huffel,et al.  The total least squares problem , 1993 .

[17]  C. L. Mallows Some Comments onCp , 1973 .

[18]  A. Satorra,et al.  Measurement Error Models , 1988 .

[19]  H. Zou,et al.  Regularization and variable selection via the elastic net , 2005 .

[20]  L. Scharf,et al.  Statistical Signal Processing: Detection, Estimation, and Time Series Analysis , 1991 .

[21]  M. Thompson Selection of Variables in Multiple Regression: Part I. A Review and Evaluation , 1978 .

[22]  Gene H. Golub,et al.  An analysis of the total least squares problem , 1980, Milestones in Matrix Computation.

[23]  G. Stewart Perturbation theory for rectangular matrix pencils , 1994 .

[24]  J. H. Wilkinson The algebraic eigenvalue problem , 1966 .

[25]  Jean-Jacques Fuchs Rectangular Pisarenko method applied to source localization , 1996, IEEE Trans. Signal Process..

[26]  Alan J. Miller Subset Selection in Regression , 1992 .

[27]  Zhi-Quan Luo,et al.  Robust adaptive beamforming using worst-case performance optimization: a solution to the signal mismatch problem , 2003, IEEE Trans. Signal Process..

[28]  Gene H. Golub,et al.  Matrix computations , 1983 .