Efficient evolution of ART neural networks

Genetic algorithms have been used to evolve several neural network architectures. In a previous effort, we introduced the evolution of three well known ART architects; Fuzzy ARTMAP (FAM), Ellipsoidal ARTMAP (EAM) and Gaussian ARTMAP (GAM). The resulting architectures were shown to achieve competitive generalization and exceptionally small size. A major concern regarding these architectures, and any evolved neural network architecture in general, is the added overhead in terms of computational time needed to produce the finally evolved network. In this paper we investigate ways of reducing this computational overhead by reducing the computations needed for the calculation of the fitness value of the evolved ART architectures. The results obtained in this paper can be directly extended to many other evolutionary neural network architectures, beyond the studied evolution of ART neural network architectures.

[1]  Shiro Usui,et al.  Mutation-based genetic neural network , 2005, IEEE Transactions on Neural Networks.

[2]  K. Dejong,et al.  An analysis of the behavior of a class of genetic adaptive systems , 1975 .

[3]  Lipo Wang,et al.  A GA-based RBF classifier with class-dependent features , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[4]  X. Yao.  Evolving Artificial Neural Networks , 1999 .

[5]  Mansooreh Mollaghasemi,et al.  Genetic Optimization of ART Neural Network Architectures , 2007, 2007 International Joint Conference on Neural Networks.

[6]  J. Fitzpatrick,et al.  Genetic Algorithms in Noisy Environments , 2005, Machine Learning.

[7]  Georgios C. Anagnostopoulos,et al.  Novel approaches in adaptive resonance theory for machine learning , 2001 .

[8]  Y.A. Dimitriadis,et al.  Safe-/spl mu/ARTMAP: a new solution for reducing category proliferation in fuzzy ARTMAP , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[9]  Konstantinos P. Ferentinos,et al.  Biological engineering applications of feedforward neural networks designed and parameterized by genetic algorithms , 2005, Neural Networks.

[10]  Mansooreh Mollaghasemi,et al.  GFAM: Evolving Fuzzy ARTMAP Neural Networks , 2007, FLAIRS.

[11]  James R. Williamson,et al.  Gaussian ARTMAP: A Neural Network for Fast Incremental Learning of Noisy Multidimensional Maps , 1996, Neural Networks.

[12]  Yaochu Jin,et al.  A comprehensive survey of fitness approximation in evolutionary computation , 2005, Soft Comput..

[13]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[14]  Georgios C. Anagnostopoulos,et al.  Exemplar-based pattern recognition via semi-supervised learning , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[15]  Kalyanmoy Deb,et al.  Genetic Algorithms, Noise, and the Sizing of Populations , 1992, Complex Syst..

[16]  Bruce A. Whitehead,et al.  Cooperative-competitive genetic evolution of radial basis function centers and widths for time series prediction , 1996, IEEE Trans. Neural Networks.

[17]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[18]  Brad L. Miller,et al.  Noise, sampling, and efficient genetic algorthms , 1997 .

[19]  Robert E. Smith,et al.  Fitness inheritance in genetic algorithms , 1995, SAC '95.

[20]  Mansooreh Mollaghasemi,et al.  m-GFAM:: An elegant approach to genetically optimize Fuzzy ARTMAP Neural Network Architectures , 2007 .

[21]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[22]  W. Mendenhall,et al.  Statistics for engineering and the sciences , 1984 .

[23]  Ah-Hwee Tan,et al.  Rule Extraction: From Neural Architecture to Symbolic Representation , 1995 .

[24]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[25]  Dorothea Heiss-Czedik,et al.  An Introduction to Genetic Algorithms. , 1997, Artificial Life.

[26]  David E. Goldberg,et al.  Evaluation relaxation using substructural information and linear estimation , 2006, GECCO '06.