Investigating a wrapper approach for selecting features using constructive neural networks

This paper investigates the problem of feature subset selection using a wrapper approach implemented using genetic algorithm and a constructive neural network. The main goal of the experiments conducted is to investigate whether the subset of features identified by the wrapper approach, implemented using the DistAl constructive neural algorithm, can also improve the accuracy of other constructive neural algorithms, namely, Tower, Tiling and Upstart algorithms. The results show that, in spite of the wrapper being directed by DistAl, the feature subsets selected can improve the accuracy of the other constructive neural algorithms as well.

[1]  S. Salzberg.  A nearest hyperrectangle learning method , 2004, Machine Learning.

[2]  Tony R. Martinez,et al.  Improved Heterogeneous Distance Functions , 1996, J. Artif. Intell. Res..

[3]  Jean-Pierre Nadal,et al.  Study of a Growth Algorithm for a Feedforward Network , 1989, Int. J. Neural Syst..

[4]  Jihoon Yang,et al.  DistAl: an inter-pattern distance-based constructive learning algorithm , 1998, 1998 IEEE International Joint Conference on Neural Networks Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98CH36227).

[5]  Huan Liu,et al.  Feature Selection for Classification , 1997, Intell. Data Anal..

[6]  Nadine N. Tschichold-Gürman.  The neural network model RuleNet and its application to mobile robot navigation , 1997, Fuzzy Sets Syst..

[7]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[8]  P. Langley.  Selection of Relevant Features in Machine Learning , 1994 .

[9]  Pat Langley,et al.  Selection of Relevant Features and Examples in Machine Learning , 1997, Artif. Intell..

[10]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[11]  Ron Kohavi,et al.  Irrelevant Features and the Subset Selection Problem , 1994, ICML.

[12]  Stephen I. Gallant,et al.  Neural network learning and expert systems , 1993 .

[13]  DistAl: An inter-pattern distance-based constructive learning algorithm , 1999, Intell. Data Anal..

[14]  Paul Douglas,et al.  Proceedings International Conference on Information Technology: Coding and Computing , 2002, Proceedings. International Conference on Information Technology: Coding and Computing.

[15]  J. Nadal,et al.  Learning in feedforward layered networks: the tiling algorithm , 1989 .

[16]  F. Smieja.  Neural network constructive algorithms: Trading generalization for learning efficiency? , 1993 .

[17]  Edoardo Amaldi,et al.  Two Constructive Methods for Designing Compact Feedforward Networks of Threshold Units , 1997, Int. J. Neural Syst..

[18]  Jihoon Yang,et al.  Feature Subset Selection Using a Genetic Algorithm , 1998, IEEE Intell. Syst..

[19]  Steven Salzberg,et al.  A Nearest Hyperrectangle Learning Method , 1991, Machine Learning.

[20]  Cristiano Cervellera,et al.  Deterministic design for neural network learning: an approach based on discrepancy , 2004, IEEE Transactions on Neural Networks.

[21]  Marcus Frean,et al.  The Upstart Algorithm: A Method for Constructing and Training Feedforward Neural Networks , 1990, Neural Computation.

[22]  Kayvan Najarian.  Learning-Based Complexity Evaluation of Radial Basis Function Networks , 2004, Neural Processing Letters.

[23]  Jihoon Yang,et al.  Pruning strategies for the MTiling constructive learning algorithm , 1997, Proceedings of International Conference on Neural Networks (ICNN'97).

[24]  Vasant G Honavar,et al.  Feature Subset Selection Using a Genetic Algorithm Feature Subset Selection Using a Genetic Algorithm , 1998 .

[25]  Constantin F. Aliferis,et al.  Towards Principled Feature Selection: Relevancy, Filters and Wrappers , 2003 .