An Adaptive Multiobjective Approach to Evolving

In this paper, we present the evolution of adaptive resonance theory (ART) neural network architectures (classifiers) using a multiobjective optimization approach. In particular, we propose the use of a multiobjective evolutionary approach to simultaneously evolve the weights and the topology of three well-known ART architectures; fuzzy ARTMAP (FAM), ellip- soidal ARTMAP (EAM), and Gaussian ARTMAP (GAM). We refer to the resulting architectures as MO-GFAM, MO-GEAM, and MO-GGAM, and collectively as MO-GART. The major advantage of MO-GART is that it produces a number of solu- tions for the classification problem at hand that have different levels of merit (accuracy on unseen data (generalization) and size (number of categories created)). MO-GART is shown to be more elegant (does not require user intervention to define the network parameters), more effective (of better accuracy and smaller size), and more efficient (faster to produce the solution networks) than other ART neural network architectures that have appeared in the literature. Furthermore, MO-GART is shown to be competitive with other popular classifiers, such as classification and regression tree (CART) and support vector machines (SVMs).

[1]  Marco Laumanns,et al.  SPEA2: Improving the strength pareto evolutionary algorithm , 2001 .

[2]  Bernhard Sendhoff,et al.  Neural network regularization and ensembling using multi-objective evolutionary algorithms , 2004, Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat. No.04TH8753).

[3]  Hideo Tanaka,et al.  Construction of fuzzy classification systems with rectangular fuzzy rules using genetic algorithms , 1994, CVPR 1994.

[4]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[5]  Georgios C. Anagnostopoulos,et al.  Novel approaches in adaptive resonance theory for machine learning , 2001 .

[6]  Mansooreh Mollaghasemi,et al.  GFAM: Evolving Fuzzy ARTMAP Neural Networks , 2007, FLAIRS.

[7]  Lalit M. Patnaik,et al.  Adaptive probabilities of crossover and mutation in genetic algorithms , 1994, IEEE Trans. Syst. Man Cybern..

[8]  Jonathan E. Fieldsend,et al.  Using unconstrained elite archives for multiobjective optimization , 2003, IEEE Trans. Evol. Comput..

[9]  Georgios C. Anagnostopoulos,et al.  Cross-validation in Fuzzy ARTMAP for large databases , 2001, Neural Networks.

[10]  Kalyanmoy Deb,et al.  Muiltiobjective Optimization Using Nondominated Sorting in Genetic Algorithms , 1994, Evolutionary Computation.

[11]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[12]  Georgios C. Anagnostopoulos,et al.  Exemplar-based pattern recognition via semi-supervised learning , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[13]  Bernhard Sendhoff,et al.  Evolutionary Multi-objective Optimization of Spiking Neural Networks , 2007, ICANN.

[14]  Hisao Ishibuchi,et al.  Three-objective genetics-based machine learning for linguistic rule extraction , 2001, Inf. Sci..

[15]  Ah-Hwee Tan,et al.  Rule Extraction: From Neural Architecture to Symbolic Representation , 1995 .

[16]  Y.A. Dimitriadis,et al.  Safe-/spl mu/ARTMAP: a new solution for reducing category proliferation in fuzzy ARTMAP , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[17]  James R. Williamson,et al.  A Constructive, Incremental-Learning Network for Mixture Modeling and Classification , 1997, Neural Computation.

[18]  Marco Laumanns,et al.  Combining Convergence and Diversity in Evolutionary Multiobjective Optimization , 2002, Evolutionary Computation.

[19]  Hisao Ishibuchi,et al.  Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems , 1997, Fuzzy Sets Syst..

[20]  Peter J. Fleming,et al.  Genetic Algorithms for Multiobjective Optimization: FormulationDiscussion and Generalization , 1993, ICGA.

[21]  Gail A. Carpenter,et al.  Distributed ARTMAP: a neural network for fast distributed supervised learning , 1998, Neural Networks.

[22]  Kalyanmoy Deb,et al.  A Fast Elitist Non-dominated Sorting Genetic Algorithm for Multi-objective Optimisation: NSGA-II , 2000, PPSN.

[23]  Hisao Ishibuchi,et al.  Multi-objective genetic local search algorithm , 1996, Proceedings of IEEE International Conference on Evolutionary Computation.

[24]  Peter J. Angeline,et al.  An evolutionary algorithm that constructs recurrent neural networks , 1994, IEEE Trans. Neural Networks.

[25]  Stephen Grossberg,et al.  Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system , 1991, Neural Networks.

[26]  Stephen Grossberg,et al.  Art 2: Self-Organization Of Stable Category Recognition Codes For Analog Input Patterns , 1988, Other Conferences.

[27]  Alex A. Freitas,et al.  Extracting comprehensible rules from neural networks via genetic algorithms , 2000, 2000 IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks. Proceedings of the First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (Cat. No.00.

[28]  Kenneth Alan De Jong,et al.  An analysis of the behavior of a class of genetic adaptive systems. , 1975 .

[29]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[30]  Yannis A. Dimitriadis,et al.  Study of distributed learning as a solution to category proliferation in Fuzzy ARTMAP based neural systems , 2003, Neural Networks.

[31]  John C. Platt,et al.  Fast training of support vector machines using sequential minimal optimization, advances in kernel methods , 1999 .

[32]  Mansooreh Mollaghasemi,et al.  Genetic Optimization of ART Neural Network Architectures , 2007, 2007 International Joint Conference on Neural Networks.

[33]  Kalyanmoy Deb,et al.  A fast and elitist multiobjective genetic algorithm: NSGA-II , 2002, IEEE Trans. Evol. Comput..

[34]  Jeffrey Horn,et al.  The Niched Pareto Genetic Algorithm 2 Applied to the Design of Groundwater Remediation Systems , 2001, EMO.

[35]  Terence C. Fogarty,et al.  Varying the Probability of Mutation in the Genetic Algorithm , 1989, ICGA.

[36]  Zbigniew Michalewicz,et al.  Adaptation in evolutionary computation: a survey , 1997, Proceedings of 1997 IEEE International Conference on Evolutionary Computation (ICEC '97).

[37]  Yaochu Jin,et al.  Dynamic Weighted Aggregation for evolutionary multi-objective optimization: why does it work and how? , 2001 .

[38]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[39]  Lothar Thiele,et al.  Multiobjective evolutionary algorithms: a comparative case study and the strength Pareto approach , 1999, IEEE Trans. Evol. Comput..

[40]  Stephen Grossberg,et al.  ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network , 1991, [1991 Proceedings] IEEE Conference on Neural Networks for Ocean Engineering.

[41]  Yannis A. Dimitriadis,et al.  Artmap: Use of Mutual Information for Category Reduction in Fuzzy Artmap , 2002 .

[42]  Robert F. Harrison,et al.  A modified fuzzy ARTMAP architecture for the approximation of noisy mappings , 1995, Neural Networks.

[43]  Jonathan E. Fieldsend,et al.  Multi-class ROC analysis from a multi-objective optimisation perspective , 2006, Pattern Recognit. Lett..

[44]  Carlos A. Coello Coello,et al.  Evolutionary multi-objective optimization: a historical view of the field , 2006, IEEE Comput. Intell. Mag..

[45]  Stephen Grossberg,et al.  Adaptive pattern classification and universal recoding: II. Feedback, expectation, olfaction, illusions , 1976, Biological Cybernetics.

[46]  Bernhard Sendhoff,et al.  Generalization Improvement in Multi-Objective Learning , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[47]  Hisao Ishibuchi,et al.  Selecting fuzzy if-then rules for classification problems using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[48]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[49]  James R. Williamson,et al.  Gaussian ARTMAP: A Neural Network for Fast Incremental Learning of Noisy Multidimensional Maps , 1996, Neural Networks.

[50]  J. Dennis,et al.  A closer look at drawbacks of minimizing weighted sums of objectives for Pareto set generation in multicriteria optimization problems , 1997 .