A hybrid model of fuzzy ARTMAP and genetic algorithm for data classification and rule extraction

A hybrid model (QFAM-GA) for data classification and rule extraction is proposed.Fuzzy ARTMAP (FAM) with Q-learning is first used for incremental learning of data.A Genetic Algorithm (GA) is then used for feature selection and rule extraction.Pruning is used to reduce the network complexity and to facilitate rule extraction.The results show QFAM-GA can provide useful if-then rule to explain its predictions. A two-stage hybrid model for data classification and rule extraction is proposed. The first stage uses a Fuzzy ARTMAP (FAM) classifier with Q-learning (known as QFAM) for incremental learning of data samples, while the second stage uses a Genetic Algorithm (GA) for rule extraction from QFAM. Given a new data sample, the resulting hybrid model, known as QFAM-GA, is able to provide prediction pertaining to the target class of the data sample as well as to give a fuzzy if-then rule to explain the prediction. To reduce the network complexity, a pruning scheme using Q-values is applied to reduce the number of prototypes generated by QFAM. A 'don't care' technique is employed to minimize the number of input features using the GA. A number of benchmark problems are used to evaluate the effectiveness of QFAM-GA in terms of test accuracy, noise tolerance, model complexity (number of rules and total rule length). The results are comparable, if not better, than many other models reported in the literature. The main significance of this research is a usable and useful intelligent model (i.e., QFAM-GA) for data classification in noisy conditions with the capability of yielding a set of explanatory rules with minimum antecedents. In addition, QFAM-GA is able to maximize accuracy and minimize model complexity simultaneously. The empirical outcome positively demonstrate the potential impact of QFAM-GA in the practical environment, i.e., providing an accurate prediction with a concise justification pertaining to the prediction to the domain users, therefore allowing domain users to adopt QFAM-GA as a useful decision support tool in assisting their decision-making processes.

[1]  Zoran Miljkovic,et al.  Neural network Reinforcement Learning for visual control of robot manipulators , 2013, Expert Syst. Appl..

[2]  Tomasz Barszcz,et al.  Hybrid System of ART and RBF Neural Networks for Classification of Vibration Signals and Operational States of Wind Turbines , 2014, ICAISC.

[3]  Aristidis Likas.  Reinforcement Learning Using the Stochastic Fuzzy Min–Max Neural Network , 2004, Neural Processing Letters.

[4]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[5]  Maizura Mokhtar,et al.  Comparing the online learning capabilities of Gaussian ARTMAP and Fuzzy ARTMAP for building energy management systems , 2013, Expert Syst. Appl..

[6]  Adel Sabry Eesa,et al.  A novel feature-selection approach based on the cuttlefish optimization algorithm for intrusion detection systems , 2015, Expert Syst. Appl..

[7]  Hisao Ishibuchi,et al.  Fuzzy Data Mining by Heuristic Rule Extraction and Multiobjective Genetic Rule Selection , 2006, 2006 IEEE International Conference on Fuzzy Systems.

[8]  Seoung Bum Kim,et al.  Sequential random k-nearest neighbor feature selection for high-dimensional data , 2015, Expert Syst. Appl..

[9]  A. Asuncion,et al.  UCI Machine Learning Repository, University of California, Irvine, School of Information and Computer Sciences , 2007 .

[10]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[11]  Mehmet Karakose,et al.  Reinforcement Learning Based Artificial Immune Classifier , 2013, TheScientificWorldJournal.

[12]  Chee Peng Lim,et al.  A Hybrid Neural Network System for Pattern Classification Tasks with Missing Features , 2005, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Chee Peng Lim,et al.  An Enhanced Fuzzy Min–Max Neural Network for Pattern Classification , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[14]  Konstantinos Blekas,et al.  A reinforcement learning approach based on the fuzzy min-max neural network , 2004, Neural Processing Letters.

[15]  Chin-Teng Lin,et al.  Reinforcement learning for an ART-based fuzzy adaptive learning control network , 1996, IEEE Trans. Neural Networks.

[16]  Mohammad A. Jaradat,et al.  Reinforcement based mobile robot navigation in dynamic environment , 2011 .

[17]  Luciano Vieira Lima,et al.  Comparing strategies for modeling students learning styles through reinforcement learning in adaptive and intelligent educational systems: An experimental analysis , 2013, Expert Syst. Appl..

[18]  Hisao Ishibuchi,et al.  Three-objective genetics-based machine learning for linguistic rule extraction , 2001, Inf. Sci..

[19]  Kay Sin Tan,et al.  A Quteishat, CP Lim,KS Tan. A modified fuzzy min-max neural network with a genetic-algorithm-based rule extractor for pattern classification. IEEE Transactions on Systems, Man, and Cybernetics-Part A:Systems and Humans. Vol 30. May 2010:641-650 , 2010 .

[20]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[21]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[22]  D. Vogiatzis,et al.  Reinforcement learning for rule extraction from a labeled dataset , 2002, Cognitive Systems Research.

[23]  S. Haykin,et al.  A Q-learning-based dynamic channel assignment technique for mobile communication systems , 1999 .

[24]  Patrick K. Simpson,et al.  Fuzzy min-max neural networks. I. Classification , 1992, IEEE Trans. Neural Networks.

[25]  R Ratcliff,et al.  Connectionist models of recognition memory: constraints imposed by learning and forgetting functions. , 1990, Psychological review.

[26]  Eyke Hüllermeier,et al.  FURIA: an algorithm for unordered fuzzy rule induction , 2009, Data Mining and Knowledge Discovery.

[27]  Guo Chen,et al.  A new approach for rule extraction of expert system based on SVM , 2014 .

[28]  Changjiu Zhou,et al.  Estimating Biped Gait Using Spline-Based Probability Distribution Function With Q-Learning , 2008, IEEE Transactions on Industrial Electronics.

[29]  Hasan Bal,et al.  Comparing performances of backpropagation and genetic algorithms in the data classification , 2011, Expert Syst. Appl..

[30]  Chee Peng Lim,et al.  Fuzzy ARTMAP dynamic decay adjustment: An improved fuzzy ARTMAP model with a conflict resolving facility , 2008, Appl. Soft Comput..

[31]  Hongbing Ji,et al.  TPPFAM: Use of threshold and posterior probability for category reduction in fuzzy ARTMAP , 2014, Neurocomputing.

[32]  Lale Özbakir,et al.  Training neural networks with harmony search algorithms for classification problems , 2012, Eng. Appl. Artif. Intell..

[33]  Tugrul U. Daim,et al.  Using artificial neural network models in stock market index prediction , 2011, Expert Syst. Appl..

[34]  Steven Salzberg,et al.  Programs for Machine Learning , 2004 .

[35]  Hyunsoo Yoon,et al.  Algorithm learning based neural network integrating feature selection and classification , 2013, Expert Syst. Appl..

[36]  Jui-Chen Huang,et al.  Remote health monitoring adoption model based on artificial neural networks , 2010, Expert Syst. Appl..

[37]  Paulo J. G. Lisboa,et al.  Orthogonal search-based rule extraction (OSRE) for trained neural networks: a practical and efficient approach , 2006, IEEE Transactions on Neural Networks.

[38]  Francisco Herrera,et al.  A first study on the noise impact in classes for Fuzzy Rule Based Classification Systems , 2010, 2010 IEEE International Conference on Intelligent Systems and Knowledge Engineering.

[39]  John Moody,et al.  Fast Learning in Networks of Locally-Tuned Processing Units , 1989, Neural Computation.

[40]  Avelino J. Gonzalez,et al.  Data-partitioning using the Hilbert space filling curves: Effect on the speed of convergence of Fuzzy ARTMAP for large database problems , 2005, Neural Networks.

[41]  Issam Dagher,et al.  An ordering algorithm for pattern presentation in fuzzy ARTMAP that tends to improve generalization performance , 1999, IEEE Trans. Neural Networks.

[42]  Amit Gupta,et al.  Generalized Analytic Rule Extraction for Feedforward Neural Networks , 1999, IEEE Trans. Knowl. Data Eng..

[43]  Jesús Alcalá-Fdez,et al.  KEEL Data-Mining Software Tool: Data Set Repository, Integration of Algorithms and Experimental Analysis Framework , 2011, J. Multiple Valued Log. Soft Comput..

[44]  Analía Amandi,et al.  A reinforcement learning approach to improve the argument selection effectiveness in argumentation-based negotiation , 2013, Expert Syst. Appl..

[45]  Rainer Matyssek,et al.  Impacts of Air Pollution and Climate Change on Forest Ecosystems — Emerging Research Needs , 2007, TheScientificWorldJournal.

[46]  Guoqiang Peter Zhang,et al.  Neural networks for classification: a survey , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[47]  Michela Antonelli,et al.  A fast and efficient multi-objective evolutionary learning scheme for fuzzy rule-based classifiers , 2014, Inf. Sci..

[48]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[49]  Ah-Hwee Tan,et al.  Rule Extraction: From Neural Architecture to Symbolic Representation , 1995 .

[50]  Chee Peng Lim,et al.  A Modified Fuzzy Min–Max Neural Network With a Genetic-Algorithm-Based Rule Extractor for Pattern Classification , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[51]  Robert F. Harrison,et al.  A modified fuzzy ARTMAP architecture for the approximation of noisy mappings , 1995, Neural Networks.

[52]  P. S. Heyns,et al.  Reconstruction of road defects and road roughness classification using Artificial Neural Networks simulation and vehicle dynamic responses: Application to experimental data , 2014 .

[53]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[54]  R. B. Gnana Jothi,et al.  Hybrid neural network for classification of graph structured data , 2015 .

[55]  Michael Georgiopoulos,et al.  Classification of noisy signals using fuzzy ARTMAP neural networks , 2001, IEEE Trans. Neural Networks.

[56]  Ali Hamzeh,et al.  Using reinforcement learning to find an optimal set of features , 2013, Comput. Math. Appl..

[57]  Francisco Herrera,et al.  Multiobjective genetic fuzzy rule selection of single granularity-based fuzzy classification rules and its interaction with the lateral tuning of membership functions , 2011, Soft Comput..

[58]  B. J. Taylor,et al.  Rule extraction as a formal method for the verification and validation of neural networks , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[59]  Mansooreh Mollaghasemi,et al.  GFAM: A Genetic Algorithm Optimization of Fuzzy ARTMAP , 2006, 2006 IEEE International Conference on Fuzzy Systems.

[60]  Peter Nemeth.  Application Possibilities of Artificial Neural Networks in the Construction Industry , 2014, 2014 International Conference on Computational Science and Computational Intelligence.

[61]  Yannis A. Dimitriadis,et al.  Study of distributed learning as a solution to category proliferation in Fuzzy ARTMAP based neural systems , 2003, Neural Networks.

[62]  Stephen Grossberg,et al.  Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system , 1991, Neural Networks.

[63]  David Hinkley,et al.  Bootstrap Methods: Another Look at the Jackknife , 2008 .

[64]  Michael McCloskey,et al.  Catastrophic Interference in Connectionist Networks: The Sequential Learning Problem , 1989 .

[65]  Adrian-Vasile Duka,et al.  Neural Network based Inverse Kinematics Solution for Trajectory Tracking of a Robotic Arm , 2014 .

[66]  Hisao Ishibuchi,et al.  Single-objective and two-objective genetic algorithms for selecting linguistic rules for pattern classification problems , 1997, Fuzzy Sets Syst..

[67]  Graham Leedham,et al.  Reinforcement learning combined with a fuzzy adaptive learning control network (FALCON-R) for pattern classification , 2005, Pattern Recognit..

[68]  James R. Williamson,et al.  Gaussian ARTMAP: A Neural Network for Fast Incremental Learning of Noisy Multidimensional Maps , 1996, Neural Networks.

[69]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[70]  P. K. Simpson,et al.  Fuzzy min-max neural networks , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.