Ranking a Random Feature for Variable and Feature Selection

We describe a feature selection method that can be applied directly to models that are linear with respect to their parameters, and indirectly to others. It is independent of the target machine. It is closely related to classical statistical hypothesis tests, but it is more intuitive, hence more suitable for use by engineers who are not statistics experts. Furthermore, some assumptions of classical tests are relaxed. The method has been used successfully in a number of applications that are briefly described.

[1]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[2]  R. H. Myers.  Classical and modern regression with applications , 1986 .

[3]  Frantz Vichot,et al.  Using Learning-based Filters to Detect Rule-based Filtering Obsolescence , 2000, RIAO.

[4]  Mathieu Stricker.  Reseaux de neurones pour le traitement automatique du langage : conception et realisation de filtres d'informations , 2000 .

[5]  Gaetan Monari.  Sélection de modèles non linéaires par "leave-one-out": étude théorique et application des réseaux de neurones au procédé de soudage par points , 1999 .

[6]  Gérard Dreyfus,et al.  Toward a Principled Methodology for Neural Network Design and Performance Evaluation in QSAR. Application to the Prediction of LogP , 1998, J. Chem. Inf. Comput. Sci..

[7]  Pascal Vincent,et al.  Kernel Matching Pursuit , 2002, Machine Learning.

[8]  D.G. Dudley,et al.  Dynamic system identification experiment design and data analysis , 1979, Proceedings of the IEEE.

[9]  J. Wolfowitz,et al.  An Introduction to the Theory of Statistics , 1951, Nature.

[10]  Hervé Stoppiglia.  Méthodes statistiques de sélection de modèles neuronaux ; applications financières et bancaires , 1997 .

[11]  Sayan Mukherjee,et al.  Feature Selection for SVMs , 2000, NIPS.

[12]  N. Bodor,et al.  Neural network studies: Part 3. Prediction of partition coefficients , 1994 .

[13]  Patrice Aknin,et al.  A new decision criterion for feature selection application to the classification of non destructive testing signatures , 1998, 9th European Signal Processing Conference (EUSIPCO 1998).

[14]  Russell Reed,et al.  Pruning algorithms-a survey , 1993, IEEE Trans. Neural Networks.

[15]  A. Bjorck,et al.  SOLVING LINEAR LEAST SQUARES PROBLEMS BY GRAM-SCHMIDT ORTHOGONALIZATION , 2005 .

[16]  Yacine Oussar,et al.  Reseaux d'ondelettes et reseaux de neurones pour la modelisation statique et dynamique de processus , 1998 .

[17]  I. J. Leontaritis,et al.  Model selection and validation methods for non-linear systems , 1987 .

[18]  L. Personnaz,et al.  The selection of neural models of nonlinear dynamical systems by statistical tests , 1994, Proceedings of IEEE Workshop on Neural Networks for Signal Processing.

[19]  Gérard Dreyfus,et al.  Local Overfitting Control via Leverages , 2002, Neural Computation.

[20]  Gérard Dreyfus,et al.  Withdrawing an example from the training set: An analytic estimation of its effect on a non-linear parameterised model , 2000, Neurocomputing.

[21]  Shang-Liang Chen,et al.  Orthogonal least squares learning algorithm for radial basis function networks , 1991, IEEE Trans. Neural Networks.

[22]  T. Söderström.  On model structure testing in system identification , 1977 .

[23]  Evelyne Tzoukermann,et al.  Information retrieval based on context distance and morphology , 1999, SIGIR '99.

[24]  Gérard Dreyfus,et al.  Initialization by selection for wavelet network training , 2000, Neurocomputing.

[25]  Å. Björck.  Solving linear least squares problems by Gram-Schmidt orthogonalization , 1967 .

[26]  Gérard Dreyfus,et al.  Two-Step Feature Selection and Neural Network Classification for the TREC-8 Routing , 1999, TREC.

[27]  D. Hoekman.  Exploring QSAR Fundamentals and Applications in Chemistry and Biology, Volume 1. Hydrophobic, Electronic and Steric Constants, Volume 2 J. Am. Chem. Soc. 1995, 117, 9782 , 1996 .