Automatic simultaneous architecture and parameter search in fuzzy neural network learning using novel variable length crossover differential evolution

The automatic simultaneous search of structure and parameters in fuzzy-neural networks is a pressing research problem. This paper introduces a novel and powerful variable-length-crossover differential evolution algorithm, vlX-DE, which is applied to ASuPFuNIS fuzzy-neural model learning, and permits simultaneous evolution of mixed-length populations of strings representing ASuPFuNIS network instances in different rules spaces. As hybrid populations of strings evolve using vlX-DE, the population gradually converges to a single rule space after which parameter search within that space proceeds till the end of the algorithm run. Search can be directed to stress either rule node economy or minimize the sum-square-error, or trade-off between these two. Tests on three benchmark problems-iris classification, CHEM classification, and Narazaki-function approximation-clearly highlight the effectiveness of the algorithm in being able to perform this simultaneous search.

[1]  Bart Kosko,et al.  Fuzzy Engineering , 1996 .

[2]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[3]  R. Storn,et al.  Differential Evolution - A simple and efficient adaptive scheme for global optimization over continuous spaces , 2004 .

[4]  Tao Jiang,et al.  Efficient and robust feature extraction by maximum margin criterion , 2003, IEEE Transactions on Neural Networks.

[5]  Nikola K. Kasabov,et al.  DENFIS: dynamic evolving neural-fuzzy inference system and its application for time-series prediction , 2002, IEEE Trans. Fuzzy Syst..

[6]  Ujjwal Maulik,et al.  Fuzzy partitioning using a real-coded variable-length genetic algorithm for pixel classification , 2003, IEEE Trans. Geosci. Remote. Sens..

[7]  Satish Kumar,et al.  Parallel Evolutionary Asymmetric Subsethood Product Fuzzy-Neural Inference System with Applications , 2006, 2006 IEEE International Conference on Fuzzy Systems.

[8]  Yinghua Lin,et al.  A new approach to fuzzy-neural system modeling , 1995, IEEE Trans. Fuzzy Syst..

[9]  James C. Bezdek,et al.  Nearest prototype classification: clustering, genetic algorithms, or random search? , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[10]  Hak-Keung Lam,et al.  Tuning of the structure and parameters of a neural network using an improved genetic algorithm , 2003, IEEE Trans. Neural Networks.

[11]  Nikhil R. Pal,et al.  Genetic programming for simultaneous feature selection and classifier design , 2006, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[12]  Anca L. Ralescu,et al.  An improved synthesis method for multilayered neural networks using qualitative knowledge , 1993, IEEE Trans. Fuzzy Syst..

[13]  Rainer Storn,et al.  Differential Evolution – A Simple and Efficient Heuristic for global Optimization over Continuous Spaces , 1997, J. Glob. Optim..

[14]  Robert Fullér,et al.  Introduction to neuro-fuzzy systems , 1999, Advances in soft computing.

[15]  Kevin Warwick,et al.  Synapsing Variable-Length Crossover: Meaningful Crossover for Variable-Length Genomes , 2007, IEEE Transactions on Evolutionary Computation.

[16]  Michio Sugeno,et al.  A fuzzy-logic-based approach to qualitative modeling , 1993, IEEE Trans. Fuzzy Syst..

[17]  Riccardo Poli,et al.  New ideas in optimization , 1999 .

[18]  Annie S. Wu,et al.  Putting More Genetics into Genetic Algorithms , 1998, Evolutionary Computation.

[19]  Tung-Kuan Liu,et al.  Tuning the structure and parameters of a neural network by using hybrid Taguchi-genetic algorithm , 2006, IEEE Trans. Neural Networks.

[20]  Nikhil R. Pal,et al.  Integrated feature analysis and fuzzy rule-based system identification in a neuro-fuzzy paradigm , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[21]  Satish Kumar,et al.  Subsethood-product fuzzy neural inference system (SuPFuNIS) , 2002, IEEE Trans. Neural Networks.

[22]  Marco Russo,et al.  FuGeNeSys-a fuzzy genetic neural system for fuzzy modeling , 1998, IEEE Trans. Fuzzy Syst..

[23]  Satish Kumar,et al.  Differential Evolution Based On-Line Feature Analysis in an Asymmetric Subsethood Product Fuzzy Neural Network , 2004, ICONIP.

[24]  Deniz Erdogmus,et al.  Feature selection in MLPs and SVMs based on maximum output information , 2004, IEEE Transactions on Neural Networks.

[25]  Peter Raeth,et al.  Book review: Fuzzy Engineering by Bart Kosko (Prentice Hall, 1997) , 1998, SGAR.

[26]  Velayutham C Shunmuga.  TOWARDS EFFECTIVE DESIGN OF NEURO FUZZY SYSTEMS , 2005 .

[27]  Francisco Herrera,et al.  Genetic Fuzzy Systems - Evolutionary Tuning and Learning of Fuzzy Knowledge Bases , 2002, Advances in Fuzzy Systems - Applications and Theory.

[28]  Satish Kumar,et al.  Asymmetric subsethood-product fuzzy neural inference system (ASuPFuNIS) , 2005, IEEE Transactions on Neural Networks.

[29]  Sanghamitra Bandyopadhyay,et al.  Pixel classification using variable string genetic algorithms with chromosome differentiation , 2001, IEEE Trans. Geosci. Remote. Sens..

[30]  Detlef Nauck,et al.  Foundations Of Neuro-Fuzzy Systems , 1997 .

[31]  Satish Kumar,et al.  Fuzzy neural inference system using mutual subsethood products with applications in medical diagnosis and control , 2001, 10th IEEE International Conference on Fuzzy Systems. (Cat. No.01CH37297).