Engineering Applications of Neural Networks (2005) Model and variable selection for nonlinear model design: some developments and methodology

The paper discusses the issues of model selection and variable (feature) selection for nonlinear modeling. Methods are described, which were designed to be simple and to involve little computational overhead, but are nevertheless generic, since they do not involve ad hoc heuristics. The principles of the methods are described, and pointers to the description of applications are provided.

[1]  Gérard Dreyfus,et al.  Reply to the Comments on Local Overfitting Control via Leverages in Jacobian Conditioning Analysis for Model Validation by I. Rivals and L. Personnaz , 2004, Neural Computation.

[2]  Sheng Chen,et al.  Orthogonal least squares methods and their application to non-linear system identification , 1989 .

[3]  Franklin A. Graybill,et al.  Introduction to The theory , 1974 .

[4]  A. Bjorck,et al.  SOLVING LINEAR LEAST SQUARES PROBLEMS BY GRAM-SCHMIDT ORTHOGONALIZATION , 2005 .

[5]  Gérard Dreyfus,et al.  Local Overfitting Control via Leverages , 2002, Neural Computation.

[6]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[7]  Gaetan Monari.  Sélection de modèles non linéaires par "leave-one-out": étude théorique et application des réseaux de neurones au procédé de soudage par points , 1999 .

[8]  Radford M. Neal,et al.  High Dimensional Classification with Bayesian Neural Networks and Dirichlet Diffusion Trees , 2006, Feature Extraction.

[9]  Å. Björck.  Solving linear least squares problems by Gram-Schmidt orthogonalization , 1967 .

[10]  Yoshua Bengio,et al.  Série Scientifique Scientific Series No Unbiased Estimator of the Variance of K-fold Cross-validation No Unbiased Estimator of the Variance of K-fold Cross-validation , 2022 .

[11]  H. J. Larson,et al.  Introduction to the Theory of Statistics , 1973 .

[12]  Jinbo Bi,et al.  Dimensionality Reduction via Sparse Support Vector Machines , 2003, J. Mach. Learn. Res..

[13]  Gérard Dreyfus,et al.  Two-Step Feature Selection and Neural Network Classification for the TREC-8 Routing , 1999, TREC.

[14]  Gérard Dreyfus,et al.  Ranking a Random Feature for Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[15]  Vladimir Vapnik,et al.  Estimation of Dependences Based on Empirical Data: Springer Series in Statistics (Springer Series in Statistics) , 1982 .

[16]  David M. Allen,et al.  The Relationship Between Variable Selection and Data Agumentation and a Method for Prediction , 1974 .

[17]  Ulrich Anders,et al.  Model selection in neural networks , 1999, Neural Networks.

[18]  Alain Rakotomamonjy,et al.  Variable Selection Using SVM-based Criteria , 2003, J. Mach. Learn. Res..

[19]  Frantz Vichot,et al.  Using Learning-based Filters to Detect Rule-based Filtering Obsolescence , 2000, RIAO.

[20]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[21]  M. Stone.  Cross‐Validatory Choice and Assessment of Statistical Predictions , 1976 .