An Incremental Parallel Particle Swarm Approach for Classification Rule Discovery from Dynamic Data

Classification is a supervised learning technique that predicts the classes of unobserved data by employing a model built from available data. One of the efficient ways to represent this predictive model is to express it as an optimal set of classification rules to provide comprehensibility and precision, simultaneously. In this paper, we propose a novel incremental parallel Particle Swarm Optimization (PSO) approach for classification rule discovery. Our proposed method separates the training data into a set of data chunks regarding the classes and extracts optimal set of classification rules for each chunk in a parallel manner. In order to extract the rules from data chunks, we introduce an incremental PSO algorithm in which the previously extracted rules are directly employed to initialize the swarm population. Moreover, in each generation of the swarm, a tournament method is employed to substitute the weak individuals with strong extracted knowledge. To support the parallelism, we assign a PSO thread for each data chunk. As soon as all the PSO threads are completed, the extracted rules are integrated into a rule-base to construct a classification model. The evaluation results of the proposed approach on six datasets suggest that the classification precision of our proposed framework is competitive with offline learning methods and is 35% faster than its counterpart offline PSO approach.

[1]  Erhan Akin,et al.  Multi-objective rule mining using a chaotic particle swarm optimization algorithm , 2009, Knowl. Based Syst..

[2]  Pinar Civicioglu,et al.  A conceptual comparison of the Cuckoo-search, particle swarm optimization, differential evolution and artificial bee colony algorithms , 2013, Artificial Intelligence Review.

[3]  Ivanoe De Falco,et al.  Facing classification problems with Particle Swarm Optimization , 2007, Appl. Soft Comput..

[4]  Wei-Chang Yeh,et al.  Novel swarm optimization for mining classification rules on thyroid gland data , 2012, Inf. Sci..

[5]  Steven Guan,et al.  An incremental approach to genetic-algorithms-based classification , 2005, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[6]  Edwin Lughofer,et al.  Evolving Fuzzy Systems - Methodologies, Advanced Concepts and Applications , 2011, Studies in Fuzziness and Soft Computing.

[7]  Tim Andersen NP-Completeness of Minimum Rule Sets , 1995 .

[8]  Alex A. Freitas,et al.  A hybrid PSO/ACO algorithm for discovering classification rules in data mining , 2008 .

[9]  A. J. Yuste,et al.  Knowledge Acquisition in Fuzzy-Rule-Based Systems With Particle-Swarm Optimization , 2010, IEEE Transactions on Fuzzy Systems.

[10]  Max Bramer,et al.  Computationally efficient induction of classification rules with the PMCRI and J-PMCRI frameworks , 2012, Knowl. Based Syst..

[11]  Aurora Trinidad Ramirez Pozo,et al.  Multiple objective particle swarm for classification-rule discovery , 2007, 2007 IEEE Congress on Evolutionary Computation.

[12]  Edwin Lughofer,et al.  Reliable All-Pairs Evolving Fuzzy Classifiers , 2013, IEEE Transactions on Fuzzy Systems.

[13]  Augusto de Almeida Prado G. Torácio,et al.  Multiobjective Particle Swarm Optimization in Classification-Rule Learning , 2009 .

[14]  Feng Qian,et al.  IPSO: An immune based PSO supervised learning system for incremental learning , 2010, 2010 8th World Congress on Intelligent Control and Automation.

[15]  Junfang Zeng,et al.  Particle Swarm Algorithm for Classification Rules Generation , 2006, Sixth International Conference on Intelligent Systems Design and Applications.

[16]  Tiago Ferra de Sousa,et al.  Particle Swarm based Data Mining Algorithms for classification tasks , 2004, Parallel Comput..

[17]  Guoqiang Peter Zhang,et al.  Neural networks for classification: a survey , 2000, IEEE Trans. Syst. Man Cybern. Part C.

[18]  Ziqiang Wang,et al.  An Efficient MA-Based Classification Rule Mining Algorithm , 2008, 2008 International Conference on Computer Science and Software Engineering.

[19]  Fan Yang,et al.  Multi-objective Rule Discovery Using the Improved Niched Pareto Genetic Algorithm , 2011, 2011 Third International Conference on Measuring Technology and Mechatronics Automation.

[20]  Jon Atli Benediktsson,et al.  Fusion of Support Vector Machines for Classification of Multisensor Data , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Kuang Yu Huang,et al.  Author ' s personal copy A hybrid particle swarm optimization approach for clustering and classification of datasets , 2011 .

[22]  Michael Kirley,et al.  Mining Classification Rules Using Evolutionary Multi-objective Algorithms , 2005, KES.

[23]  Chunjie Zhou,et al.  An efficient SFL-based classification rule mining algorithm , 2008, 2008 IEEE International Symposium on IT in Medicine and Education.

[24]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[25]  Ziqiang Wang,et al.  A PSO-Based Classification Rule Mining Algorithm , 2009, ICIC.

[26]  Aiko M. Hormann,et al.  Programs for Machine Learning. Part I , 1962, Inf. Control..

[27]  Alex A. Freitas,et al.  A survey of evolutionary algorithms for data mining and knowledge discovery , 2003 .

[28]  Derya Birant,et al.  An incremental genetic algorithm for classification and sensitivity analysis of its parameters , 2011, Expert Syst. Appl..

[29]  Simone A. Ludwig,et al.  Discrete Particle Swarm Optimization with local search strategy for Rule Classification , 2012, 2012 Fourth World Congress on Nature and Biologically Inspired Computing (NaBIC).

[30]  Ziqiang Wang,et al.  Classification Rule Mining Based on Particle Swarm Optimization , 2006, RSKT.

[31]  Rajib Mall,et al.  Application of elitist multi-objective genetic algorithm for classification rule generation , 2008, Appl. Soft Comput..

[32]  William W. Cohen Fast Effective Rule Induction , 1995, ICML.

[33]  Bahriye Akay,et al.  A study on particle swarm optimization and artificial bee colony algorithms for multilevel thresholding , 2013, Appl. Soft Comput..

[34]  Ivanoe De Falco,et al.  Discovering interesting classification rules with genetic programming , 2002, Appl. Soft Comput..

[35]  M. A. Khanesar,et al.  A novel binary particle swarm optimization , 2007, 2007 Mediterranean Conference on Control & Automation.

[36]  Aurora Pozo,et al.  Mining Rules: A Parallel Multiobjective Particle Swarm Optimization Approach , 2009 .

[37]  Robert Sabourin,et al.  Dynamic multi-objective evolution of classifier ensembles for video face recognition , 2013, Appl. Soft Comput..

[38]  Li-Chen Fu,et al.  A two-phase evolutionary algorithm for multiobjective mining of classification rules , 2010, IEEE Congress on Evolutionary Computation.

[39]  Donald E. Grierson,et al.  Comparison among five evolutionary-based optimization algorithms , 2005, Adv. Eng. Informatics.

[40]  Dervis Karaboga,et al.  Artificial bee colony data miner (ABC-Miner) , 2011, 2011 International Symposium on Innovations in Intelligent Systems and Applications.

[41]  Xin Yao,et al.  A novel evolutionary data mining algorithm with applications to churn prediction , 2003, IEEE Trans. Evol. Comput..

[42]  Chih-Chuan Chen,et al.  A Multi-objective Particle Swarm Optimization Algorithm for Rule Discovery , 2007, Third International Conference on Intelligent Information Hiding and Multimedia Signal Processing (IIH-MSP 2007).

[43]  Alex Alves Freitas,et al.  Multiple pheromone types and other extensions to the Ant-Miner classification rule discovery algorithm , 2011, Swarm Intelligence.

[44]  Sung-Bae Cho,et al.  Multi-objective Classification Rule Mining Using Gene Expression Programming , 2008, 2008 Third International Conference on Convergence and Hybrid Information Technology.