Probabilistic ensemble Fuzzy ARTMAP optimization using hierarchical parallel genetic algorithms

Abstract In this study, a comprehensive methodology for overcoming the design problem of the Fuzzy ARTMAP neural network is proposed. The issues addressed are the sequence of training data for supervised learning and optimum parameter tuning for parameters such as baseline vigilance. A genetic algorithm search heuristic was chosen to solve this multi-objective optimization problem. To further augment the ARTMAP’s pattern classification ability, multiple ARTMAPs were optimized via genetic algorithm and assembled into a classifier ensemble. An optimal ensemble was realized by the inter-classifier diversity of its constituents. This was achieved by mitigating convergence in the genetic algorithms by employing a hierarchical parallel architecture. The best-performing classifiers were then combined in an ensemble, using probabilistic voting for decision combination. This study also integrated the disparate methods to operate within a single framework, which is the proposed novel method for creating an optimum classifier ensemble configuration with minimum user intervention. The methodology was benchmarked using popular data sets from UCI machine learning repository.

[1]  Robert Sabourin,et al.  Evolution of heterogeneous ensembles through dynamic particle swarm optimization for video-based face recognition , 2012, Pattern Recognit..

[2]  MaLin,et al.  Empirical analysis of support vector machine ensemble classifiers , 2009 .

[3]  Robert Sabourin,et al.  Classification system optimization with multi-objective genetic algorithms , 2006 .

[4]  Tin Kam Ho,et al.  The Random Subspace Method for Constructing Decision Forests , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  James R. Williamson,et al.  Gaussian ARTMAP: A Neural Network for Fast Incremental Learning of Noisy Multidimensional Maps , 1996, Neural Networks.

[6]  Yannis A. Dimitriadis,et al.  Artmap: Use of Mutual Information for Category Reduction in Fuzzy Artmap , 2002 .

[7]  Hisao Ishibuchi,et al.  Evolutionary Multiobjective Optimization for Generating an Ensemble of Fuzzy Rule-Based Classifiers , 2003, GECCO.

[8]  Jihoon Yang,et al.  Feature Subset Selection Using a Genetic Algorithm , 1998, IEEE Intell. Syst..

[9]  Bernard Zenko,et al.  Is Combining Classifiers Better than Selecting the Best One , 2002, ICML.

[10]  OpitzDavid,et al.  Popular ensemble methods , 1999 .

[11]  Maizura Mokhtar,et al.  Comparing the online learning capabilities of Gaussian ARTMAP and Fuzzy ARTMAP for building energy management systems , 2013, Expert Syst. Appl..

[12]  R. Tibshirani,et al.  Classification by Set Cover: The Prototype Vector Machine , 2009, 0908.2284.

[13]  Héctor Allende,et al.  Behavior analysis of neural network ensemble algorithm on a virtual machine cluster , 2011, Neural Computing and Applications.

[14]  Sultan Noman Qasem,et al.  Multi-objective hybrid evolutionary algorithms for radial basis function neural network design , 2012, Knowl. Based Syst..

[15]  Mong-Li Lee,et al.  SNNB: A Selective Neighborhood Based Naïve Bayes for Lazy Learning , 2002, PAKDD.

[16]  Luiz Eduardo Soares de Oliveira,et al.  Feature Selection for Ensembles Using the Multi-Objective Optimization Approach , 2006, Multi-Objective Machine Learning.

[17]  David W. Opitz,et al.  Feature Selection for Ensembles , 1999, AAAI/IAAI.

[18]  Anne M. P. Canuto,et al.  Using ARTMAP-Based Ensemble Systems Designed by Three Variants of Boosting , 2008, ICANN.

[19]  Chee Peng Lim,et al.  Evolutionary Fuzzy ARTMAP Neural Networks and their Applications to Fault Detection and Diagnosis , 2010, Neural Processing Letters.

[20]  Issam Dagher,et al.  An ordering algorithm for pattern presentation in fuzzy ARTMAP that tends to improve generalization performance , 1999, IEEE Trans. Neural Networks.

[21]  Yaochu Jin,et al.  Multi-Objective Machine Learning , 2006, Studies in Computational Intelligence.

[22]  Hongbing Ji,et al.  TPPFAM: Use of threshold and posterior probability for category reduction in fuzzy ARTMAP , 2014, Neurocomputing.

[23]  William Nick Street,et al.  Ensemble Pruning Via Semi-definite Programming , 2006, J. Mach. Learn. Res..

[24]  Henrik Boström,et al.  Ensemble member selection using multi-objective optimization , 2009, 2009 IEEE Symposium on Computational Intelligence and Data Mining.

[25]  Chih-Hung Wu,et al.  A Novel hybrid genetic algorithm for kernel function and parameter optimization in support vector regression , 2009, Expert Syst. Appl..

[26]  Chee Peng Lim,et al.  Condition monitoring of induction motors: A review and an application of an ensemble of hybrid intelligent models , 2014, Expert Syst. Appl..

[27]  Amr Badr,et al.  Evolutionary Fuzzy ARTMAP Approach for Breast Cancer Diagnosis , 2011 .

[28]  Julian F. Miller,et al.  Genetic and Evolutionary Computation — GECCO 2003 , 2003, Lecture Notes in Computer Science.

[29]  D. Opitz,et al.  Popular Ensemble Methods: An Empirical Study , 1999, J. Artif. Intell. Res..

[30]  F. Wilcoxon.  Individual Comparisons by Ranking Methods , 1945 .

[31]  Andrew D. Ball,et al.  An application to transient current signal based induction motor fault diagnosis of Fourier-Bessel expansion and simplified fuzzy ARTMAP , 2013, Expert Syst. Appl..

[32]  Xin Yao,et al.  Evolutionary ensembles with negative correlation learning , 2000, IEEE Trans. Evol. Comput..

[33]  Araceli Sanchis,et al.  Genetic Approach for Optimizing Ensembles of Classifiers , 2008, FLAIRS.

[34]  Mansooreh Mollaghasemi,et al.  MO-GART: Multiobjective genetic ART architectures , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[35]  Xudong Zhao,et al.  Reliable fault diagnosis method using ensemble fuzzy ARTMAP based on improved Bayesian belief method , 2014, Neurocomputing.

[36]  Lin Ma,et al.  Empirical analysis of support vector machine ensemble classifiers , 2009, Expert Syst. Appl..

[37]  Chu Kiong Loo,et al.  Accurate and reliable diagnosis and classification using probabilistic ensemble simplified fuzzy ARTMAP , 2005, IEEE Transactions on Knowledge and Data Engineering.

[38]  Anne M P Canuto,et al.  A Reinforcement-based Mechanism to Select Features for Classifiers in Ensemble Systems , 2011 .

[39]  Pádraig Cunningham,et al.  Overfitting and Diversity in Classification Ensembles based on Feature Selection , 2000 .

[40]  Zheru Chi,et al.  Intelligent characterization and evaluation of yarn surface appearance using saliency map analysis, wavelet transform and fuzzy ARTMAP neural network , 2012, Expert Syst. Appl..

[41]  Grigorios Tsoumakas,et al.  Ensemble Pruning Using Reinforcement Learning , 2006, SETN.

[42]  Ramaswamy Palaniappan,et al.  Using genetic algorithm to select the presentation order of training patterns that improves simplified fuzzy ARTMAP classification performance , 2009, Appl. Soft Comput..

[43]  Jaewan Lee,et al.  Agent-Based Approach to Distributed Ensemble Learning of Fuzzy ARTMAP Classifiers , 2007, KES-AMSTA.

[44]  Witold Pedrycz,et al.  A new selective neural network ensemble with negative correlation , 2012, Applied Intelligence.

[45]  Mansooreh Mollaghasemi,et al.  GFAM: A Genetic Algorithm Optimization of Fuzzy ARTMAP , 2006, 2006 IEEE International Conference on Fuzzy Systems.

[46]  Araceli Sanchis,et al.  A new artificial neural network ensemble based on feature selection and class recoding , 2012, Neural Computing and Applications.

[47]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[48]  Padraig Cunningham,et al.  Using Diversity in Preparing Ensembles of Classifiers Based on Different Feature Subsets to Minimize Generalization Error , 2001, ECML.

[49]  Luiz Eduardo Soares de Oliveira,et al.  Particle Swarm Optimization of Fuzzy ARTMAP Parameters , 2006, The 2006 IEEE International Joint Conference on Neural Network Proceedings.

[50]  Gail A. Carpenter,et al.  Biased ART: A neural architecture that shifts attention toward previously disregarded features following an incorrect prediction , 2010, Neural Networks.

[51]  Steven J. Simske,et al.  Performance analysis of pattern classifier combination by plurality voting , 2003, Pattern Recognit. Lett..

[52]  Erik D. Goodman,et al.  The hierarchical fair competition (HFC) model for parallel evolutionary algorithms , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[53]  Véra Kůrková,et al.  Artificial Neural Networks - ICANN 2008 , 18th International Conference, Prague, Czech Republic, September 3-6, 2008, Proceedings, Part I , 2008, ICANN.

[54]  Robert Sabourin,et al.  Overfitting cautious selection of classifier ensembles with genetic algorithms , 2009, Inf. Fusion.

[55]  Chee Peng Lim,et al.  A hybrid intelligent system for medical data classification , 2014, Expert Syst. Appl..