Evolutionary computation for the automated design of category functions for fuzzy ART: an initial exploration

Fuzzy Adaptive Resonance Theory (ART) is a classic unsupervised learning algorithm. Its performance on a particular clustering problem is sensitive to the suitability of the category function for said problem. However, classic Fuzzy ART employs a fixed category function and thus is unable to benefit from the potential to adjust its category function. This paper presents an exploration into employing evolutionary computation for the automated design of category functions to obtain significantly enhanced Fuzzy ART performance through tailoring to specific problem classes. We employ a genetic programming powered hyper-heuristic approach where the category functions are constructed from a set of primitives constituting those of the original Fuzzy ART category function as well as additional hand-selected primitives. Results are presented for a set of experiments on benchmark classification tasks from the UCI Machine Learning Repository demonstrating that tailoring Fuzzy ART's category function can achieve statistically significant superior performance on the testing datasets in stratified 10-fold cross-validation procedures. We conclude with discussing the results and placing them in the context of being a first step towards automating the design of entirely new forms of ART.

[1]  Mansooreh Mollaghasemi,et al.  An Adaptive Multiobjective Approach to Evolving ART Architectures , 2010, IEEE Transactions on Neural Networks.

[2]  David J. Montana,et al.  Strongly Typed Genetic Programming , 1995, Evolutionary Computation.

[3]  James R. Williamson,et al.  Gaussian ARTMAP: A Neural Network for Fast Incremental Learning of Noisy Multidimensional Maps , 1996, Neural Networks.

[4]  Himansu Sekhar Behera,et al.  Evolving Low Complex Higher Order Neural Network Based Classifiers for Medical Data Classification , 2016 .

[5]  D. Wunsch,et al.  Multiclass Cancer Classification Using Semisupervised Ellipsoid ARTMAP and Particle Swarm Optimization with Gene Expression Data , 2007, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[6]  John R. Koza,et al.  Genetic programming - on the programming of computers by means of natural selection , 1993, Complex adaptive systems.

[7]  Stephen Grossberg,et al.  ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network , 1991, [1991 Proceedings] IEEE Conference on Neural Networks for Ocean Engineering.

[8]  S. Grossberg.  How does a brain build a cognitive code , 1980 .

[9]  Georgios C. Anagnostopoulos,et al.  Ellipsoid ART and ARTMAP for incremental clustering and classification , 2001, IJCNN'01. International Joint Conference on Neural Networks. Proceedings (Cat. No.01CH37222).

[10]  Mansooreh Mollaghasemi,et al.  AG-ART: An adaptive approach to evolving ART architectures , 2009, Neurocomputing.

[11]  Stephen Grossberg,et al.  Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system , 1991, Neural Networks.

[12]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[13]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[14]  Daniel R. Tauritz,et al.  A Comparison of Genetic Programming Variants for Hyper-Heuristics , 2015, GECCO.

[15]  Marzuki Khalid,et al.  Evolutionary Fuzzy ARTMAP Neural Networks for Classification of Semiconductor Defects , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[16]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[17]  Michel Gendreau,et al.  Hyper-heuristics: a survey of the state of the art , 2013, J. Oper. Res. Soc..

[18]  Ian H. Witten,et al.  The WEKA data mining software: an update , 2009, SKDD.

[19]  L. Hubert,et al.  Comparing partitions , 1985 .

[20]  Xin Yao,et al.  Evolving artificial neural networks , 1999, Proc. IEEE.

[21]  Hugo Terashima-Marín,et al.  A Neuro-evolutionary Hyper-heuristic Approach for Constraint Satisfaction Problems , 2015, Cognitive Computation.

[22]  Rong Li,et al.  Multi-objective memetic evolution of ART-based classifiers , 2010, IEEE Congress on Evolutionary Computation.

[23]  Ashraf M. Abdelbar,et al.  Biclustering ARTMAP collaborative filtering recommender system , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[24]  Rui Xu,et al.  BARTMAP: A viable structure for biclustering , 2011, Neural Networks.

[25]  Marc Parizeau,et al.  DEAP: evolutionary algorithms made easy , 2012, J. Mach. Learn. Res..

[26]  Gail A. Carpenter,et al.  Fuzzy ART Choice Functions , 1993 .