Feature Selection Based on Sensitivity Analysis

In this paper an incremental version of the ANOVA and Functional Networks Feature Selection (AFN-FS) method is presented. This new wrapper method (IAFN-FS) is based on an incremental functional decomposition, thus eliminating the main drawback of the basic method: the exponential complexity of the functional decomposition. This complexity limited its scope of applicability, being only applicable to datasets with a relatively small number of features. The performance of the incremental version of the method was tested against several real data sets. The results show that IAFN-FS outperforms the accuracy obtained by other standard and novel feature selection methods, using a small set of features.

[1]  Masoud Nikravesh,et al.  Feature Extraction - Foundations and Applications , 2006, Feature Extraction.

[2]  Amparo Alonso-Betanzos,et al.  A Very Fast Learning Method for Neural Networks Based on Sensitivity Analysis , 2006, J. Mach. Learn. Res..

[3]  Amparo Alonso-Betanzos,et al.  Functional Network Topology Learning and Sensitivity Analysis Based on ANOVA Decomposition , 2007, Neural Computation.

[4]  Ron Kohavi,et al.  Wrappers for Feature Subset Selection , 1997, Artif. Intell..

[5]  Amparo Alonso-Betanzos,et al.  Functional Networks and Analysis of Variance for Feature Selection , 2006, IDEAL.

[6]  Isabelle Guyon,et al.  An Introduction to Variable and Feature Selection , 2003, J. Mach. Learn. Res..

[7]  Sreeram Ramakrishnan,et al.  A hybrid approach for feature subset selection using neural networks and ant colony optimization , 2007, Expert Syst. Appl..

[8]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[9]  Masoud Nikravesh,et al.  Feature Extraction: Foundations and Applications (Studies in Fuzziness and Soft Computing) , 2006 .

[10]  Stefano Tarantola,et al.  Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models , 2004 .

[11]  I. Sobola,et al.  Global sensitivity indices for nonlinear mathematical models and their Monte Carlo estimates , 2001 .

[12]  Emilio Corchado,et al.  Intelligent Data Engineering and Automated Learning - IDEAL 2006, 7th International Conference, Burgos, Spain, September 20-23, 2006, Proceedings , 2006, IDEAL.

[13]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[14]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .