Parallel Distributed Hybrid Fuzzy GBML Models With Rule Set Migration and Training Data Rotation

We propose a parallel distributed model of a hybrid fuzzy genetics-based machine learning (GBML) algorithm to drastically decrease its computation time. Our hybrid algorithm has a Pittsburgh-style GBML framework where a rule set is coded as an individual. A Michigan-style rule-generation mechanism is used as a kind of local search. Our parallel distributed model is an island model where a population of individuals is divided into multiple islands. Training data are also divided into multiple subsets. The main feature of our model is that a different training data subset is assigned to each island. The assigned training data subsets are periodically rotated over the islands. The best rule set in each island also migrates periodically. We demonstrate through computational experiments that our model decreases the computation time of the hybrid fuzzy GBML algorithm by an order or two of magnitude using seven parallel processors without severely degrading the generalization ability of obtained fuzzy rule-based classifiers. We also examine the effects of the training data rotation and the rule set migration on the search ability of our model.

[1]  El-Ghazali Talbi,et al.  ParadisEO: A Framework for the Reusable Design of Parallel and Distributed Metaheuristics , 2004, J. Heuristics.

[2]  Mohammad Saniee Abadeh,et al.  A fuzzy classification system based on Ant Colony Optimization for diabetes disease diagnosis , 2011, Expert Syst. Appl..

[3]  Beatrice Lazzerini,et al.  Multi-objective genetic fuzzy classifiers for imbalanced and cost-sensitive datasets , 2010, Soft Comput..

[4]  Enrique Alba,et al.  A multi-GPU implementation of a Cellular Genetic Algorithm , 2010, IEEE Congress on Evolutionary Computation.

[5]  Hisao Ishibuchi,et al.  Classification and modeling with linguistic information granules - advanced approaches to linguistic data mining , 2004, Advanced information processing.

[6]  Hisao Ishibuchi,et al.  Performance evaluation of fuzzy classifier systems for multidimensional pattern classification problems , 1999, IEEE Trans. Syst. Man Cybern. Part B.

[7]  Jesús Alcalá-Fdez,et al.  A Fuzzy Association Rule-Based Classification Model for High-Dimensional Problems With Genetic Rule Selection and Lateral Tuning , 2011, IEEE Transactions on Fuzzy Systems.

[8]  Enrique Alba,et al.  Parallelism and evolutionary algorithms , 2002, IEEE Trans. Evol. Comput..

[9]  Hisao Ishibuchi,et al.  Parallel Distributed Implementation of Genetics-Based Machine Learning for Fuzzy Classifier Design , 2010, SEAL.

[10]  Huan Liu,et al.  On Issues of Instance Selection , 2002, Data Mining and Knowledge Discovery.

[11]  María José del Jesús,et al.  KEEL: a software tool to assess evolutionary algorithms for data mining problems , 2008, Soft Comput..

[12]  Hisao Ishibuchi,et al.  Training Data Subdivision and Periodical Rotation in Hybrid Fuzzy Genetics-Based Machine Learning , 2011, 2011 10th International Conference on Machine Learning and Applications and Workshops.

[13]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[14]  Xin Yao,et al.  Experimental study on population-based incremental learning algorithms for dynamic optimization problems , 2005, Soft Comput..

[15]  Francisco Herrera,et al.  Stratification for scaling up evolutionary prototype selection , 2005, Pattern Recognit. Lett..

[16]  Agostinho C. Rosa,et al.  A comparative study on the performance of dissortative mating and immigrants-based strategies for evolutionary dynamic optimization , 2011, Inf. Sci..

[17]  Oscar Castillo,et al.  Optimization of interval type-2 fuzzy logic controllers using evolutionary algorithms , 2011, Soft Comput..

[18]  Enrique Alba,et al.  Parallel Genetic Algorithms , 2011, Studies in Computational Intelligence.

[19]  Yaochu Jin,et al.  Multi-Objective Machine Learning , 2006, Studies in Computational Intelligence.

[20]  Hisao Ishibuchi,et al.  Multiobjective Genetic Fuzzy Systems: Review and Future Research Directions , 2007, 2007 IEEE International Fuzzy Systems Conference.

[21]  Jürgen Branke,et al.  Multiswarms, exclusion, and anti-convergence in dynamic environments , 2006, IEEE Transactions on Evolutionary Computation.

[22]  Francisco Herrera,et al.  A genetic tuning to improve the performance of Fuzzy Rule-Based Classification Systems with Interval-Valued Fuzzy Sets: Degree of ignorance and lateral position , 2011, Int. J. Approx. Reason..

[23]  Oscar Castillo,et al.  A review on the design and optimization of interval type-2 fuzzy controllers , 2012, Appl. Soft Comput..

[24]  María José del Jesús,et al.  A proposal on reasoning methods in fuzzy rule-based classification systems , 1999, Int. J. Approx. Reason..

[25]  D.E. Goldberg,et al.  Classifier Systems and Genetic Algorithms , 1989, Artif. Intell..

[26]  Hisao Ishibuchi,et al.  Analysis of interpretability-accuracy tradeoff of fuzzy systems by multiobjective fuzzy genetics-based machine learning , 2007, Int. J. Approx. Reason..

[27]  Jaume Bacardit,et al.  Speeding up the evaluation of evolutionary learning systems using GPGPUs , 2010, GECCO '10.

[28]  Hisao Ishibuchi,et al.  Rule weight specification in fuzzy rule-based classification systems , 2005, IEEE Transactions on Fuzzy Systems.

[29]  María José del Jesús,et al.  NMEEF-SD: Non-dominated Multiobjective Evolutionary Algorithm for Extracting Fuzzy Rules in Subgroup Discovery , 2010, IEEE Transactions on Fuzzy Systems.

[30]  Francisco Herrera,et al.  Genetics-Based Machine Learning for Rule Induction: State of the Art, Taxonomy, and Comparative Study , 2010, IEEE Transactions on Evolutionary Computation.

[31]  Shigeyoshi Tsutsui,et al.  An analytical study of GPU computation for solving QAPs by parallel evolutionary computation with independent run , 2010, IEEE Congress on Evolutionary Computation.

[32]  Martin V. Butz,et al.  Speeding-Up Pittsburgh Learning Classifier Systems: Modeling Time and Accuracy , 2004, PPSN.

[33]  Francisco Herrera,et al.  On the combination of evolutionary algorithms and stratified strategies for training set selection in data mining , 2006, Appl. Soft Comput..

[34]  Dr. Alex A. Freitas.  Data Mining and Knowledge Discovery with Evolutionary Algorithms , 2002, Natural Computing Series.

[35]  Hisao Ishibuchi,et al.  Fuzzy data mining: effect of fuzzy discretization , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[36]  H. Ishibuchi,et al.  Distributed representation of fuzzy rules and its application to pattern classification , 1992 .

[37]  Panos M. Pardalos,et al.  hGA: Hybrid genetic algorithm in fuzzy rule-based classification systems for high-dimensional problems , 2012, Appl. Soft Comput..

[38]  Carlos Cruz,et al.  Optimization in dynamic environments: a survey on problems, methods and measures , 2011, Soft Comput..

[39]  Francisco Herrera,et al.  Improving the performance of fuzzy rule-based classification systems with interval-valued fuzzy sets and genetic amplitude tuning , 2010, Inf. Sci..

[40]  Francisco Herrera,et al.  Evolutionary stratified training set selection for extracting classification rules with trade off precision-interpretability , 2007, Data Knowl. Eng..

[41]  M. Montaz Ali,et al.  Population set-based global optimization algorithms: some modifications and numerical studies , 2004, Comput. Oper. Res..

[42]  Hisao Ishibuchi,et al.  Parallel distributed genetic fuzzy rule selection , 2008, Soft Comput..

[43]  Hisao Ishibuchi,et al.  Use of very small training data subsets in parallel distributed genetic fuzzy rule selection , 2010, 2010 4th International Workshop on Genetic and Evolutionary Fuzzy Systems (GEFS).

[44]  Hisao Ishibuchi,et al.  Fuzzy rule selection by multi-objective genetic local search algorithms and rule evaluation measures in data mining , 2004, Fuzzy Sets Syst..

[45]  Hisao Ishibuchi,et al.  Voting in fuzzy rule-based systems for pattern classification problems , 1999, Fuzzy Sets Syst..

[46]  Fevrier Valdez,et al.  Comparative study of bio-inspired algorithms applied to the optimization of type-1 and type-2 fuzzy controllers for an autonomous mobile robot , 2012, Inf. Sci..

[47]  Edmund K. Burke,et al.  Improving the scalability of rule-based evolutionary learning , 2009, Memetic Comput..

[48]  Antonio J. Rivera,et al.  GP-COACH: Genetic Programming-based learning of COmpact and ACcurate fuzzy rule-based classification systems for High-dimensional problems , 2010, Inf. Sci..

[49]  William B. Langdon,et al.  Graphics processing units and genetic programming: an overview , 2011, Soft Comput..

[50]  Nicolas Lachiche,et al.  Coarse grain parallelization of evolutionary algorithms on GPGPU cards with EASEA , 2009, GECCO.

[51]  Jürgen Branke,et al.  Evolutionary optimization in uncertain environments-a survey , 2005, IEEE Transactions on Evolutionary Computation.

[52]  Hisao Ishibuchi,et al.  Selecting fuzzy if-then rules for classification problems using genetic algorithms , 1995, IEEE Trans. Fuzzy Syst..

[53]  Hisao Ishibuchi,et al.  Hybridization of fuzzy GBML approaches for pattern classification problems , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[54]  Tzung-Pei Hong,et al.  Trade-off Between Computation Time and Number of Rules for Fuzzy Mining from Quantitative Data , 2001, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[55]  Bernhard Sendhoff,et al.  Pareto-Based Multiobjective Machine Learning: An Overview and Case Studies , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[56]  Francisco Herrera,et al.  A study of statistical techniques and performance measures for genetics-based machine learning: accuracy and interpretability , 2009, Soft Comput..

[57]  Volker Nissen,et al.  On the robustness of population-based versus point-based optimization in the presence of noise , 1998, IEEE Trans. Evol. Comput..

[58]  Peter J. Bentley,et al.  Fast bio-inspired computation using a GPU-based systemic computer , 2010, Parallel Comput..

[59]  Jaume Bacardit,et al.  Performance and Efficiency of Memetic Pittsburgh Learning Classifier Systems , 2009, Evolutionary Computation.