A Constructive Data Classification Version of the Particle Swarm Optimization Algorithm

The particle swarm optimization algorithm was originally introduced to solve continuous parameter optimization problems. It was soon modified to solve other types of optimization tasks and also to be applied to data analysis. In the latter case, however, there are few works in the literature that deal with the problem of dynamically building the architecture of the system. This paper introduces new particle swarm algorithms specifically designed to solve classification problems. The first proposal, named Particle Swarm Classifier (PSClass), is a derivation of a particle swarm clustering algorithm and its architecture, as in most classifiers, is pre-defined. The second proposal, named Constructive Particle Swarm Classifier (cPSClass), uses ideas from the immune system to automatically build the swarm. A sensitivity analysis of the growing procedure of cPSClass and an investigation into a proposed pruning procedure for this algorithm are performed. The proposals were applied to a wide range of databases from the literature and the results show that they are competitive in relation to other approaches, with the advantage of having a dynamically constructed architecture.

[1]  Alex Alves Freitas,et al.  A hybrid particle swarm/ant colony algorithm for the classification of hierarchical biological data , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[2]  W. Marsden.  I and J , 2012 .

[3]  Jonathan Timmis,et al.  Artificial Immune Systems: A New Computational Intelligence Approach , 2003 .

[4]  Quan Pan,et al.  Dynamic Population Size Based Particle Swarm Optimization , 2007, ISICA.

[5]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[6]  Tiago Ferra de Sousa,et al.  Particle Swarm based Data Mining Algorithms for classification tasks , 2004, Parallel Comput..

[7]  Ziqiang Wang,et al.  A PSO-Based Classification Rule Mining Algorithm , 2009, ICIC.

[8]  R. Suganya,et al.  Data Mining Concepts and Techniques , 2010 .

[9]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .

[10]  Gary G. Yen,et al.  Dynamic population strategy assisted Particle Swarm Optimization , 2003, Proceedings of the 2003 IEEE International Symposium on Intelligent Control.

[11]  Ajith Abraham,et al.  Fuzzy C-means and fuzzy swarm for fuzzy clustering problem , 2011, Expert Syst. Appl..

[12]  Inés María Galván,et al.  A comparison between the Pittsburgh and Michigan approaches for the binary PSO algorithm , 2005, 2005 IEEE Congress on Evolutionary Computation.

[13]  Wei Kong,et al.  Hybrid particle swarm optimization and tabu search approach for selecting genes for tumor classification using gene expression data , 2008, Comput. Biol. Chem..

[14]  Marco Dorigo,et al.  Swarm intelligence: from natural to artificial systems , 1999 .

[15]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[16]  E. Bonabeau,et al.  Self-organization in social insects. , 1997, Trends in ecology & evolution.

[17]  F. Burnet.  The clonal selection theory of acquired immunity , 1959 .

[18]  Gustavo E. A. P. A. Batista,et al.  An analysis of four missing data treatment methods for supervised learning , 2003, Appl. Artif. Intell..

[19]  Kuang Yu Huang,et al.  Author ' s personal copy A hybrid particle swarm optimization approach for clustering and classification of datasets , 2011 .

[20]  Teuvo Kohonen,et al.  The self-organizing map , 1990 .

[21]  Andries P. Engelbrecht,et al.  Dynamic Clustering using Particle Swarm Optimization with Application in Unsupervised Image Classification , 2007 .

[22]  Christopher R. Westphal,et al.  Data Mining Solutions: Methods and Tools for Solving Real-World Problems , 1998 .

[23]  Teuvo Kohonen,et al.  Improved versions of learning vector quantization , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[24]  Taher Niknam,et al.  A New Evolutionary Algorithm for Cluster Analysis , 2008 .

[25]  Rory A. Fisher,et al.  Statistical methods and scientific inference. , 1957 .

[26]  ChunXia Zhao,et al.  Particle swarm optimization with adaptive population size and its application , 2009, Appl. Soft Comput..

[27]  Inés María Galván,et al.  AMPSO: A New Particle Swarm Method for Nearest Neighborhood Classification , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Jonathan Timmis,et al.  Application areas of AIS: The past, the present and the future , 2008, Appl. Soft Comput..

[29]  Jinxin Dong,et al.  A New Algorithm for Clustering Based on Particle Swarm Optimization and K-means , 2009, 2009 International Conference on Artificial Intelligence and Computational Intelligence.

[30]  Richard G. Brereton,et al.  Learning Vector Quantization for Multiclass Classification: Application to Characterization of Plastics , 2007, J. Chem. Inf. Model..

[31]  Tong Heng Lee,et al.  Evolutionary algorithms with dynamic population size and local exploration for multiobjective optimization , 2001, IEEE Trans. Evol. Comput..

[32]  Zbigniew Michalewicz,et al.  Evolutionary Computation 2 : Advanced Algorithms and Operators , 2000 .

[33]  W. Pitts,et al.  A Logical Calculus of the Ideas Immanent in Nervous Activity (1943) , 2021, Ideas That Created the Future.

[34]  M. Friedman.  The Use of Ranks to Avoid the Assumption of Normality Implicit in the Analysis of Variance , 1937 .

[35]  Ganapati Panda,et al.  Particle swarm optimized multiple regression linear model for data classification , 2009, Appl. Soft Comput..

[36]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[37]  S. Shapiro,et al.  An Analysis of Variance Test for Normality (Complete Samples) , 1965 .

[38]  Andries Petrus Engelbrecht,et al.  Data clustering using particle swarm optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[39]  Leandro Nunes de Castro,et al.  Fundamentals of natural computing: an overview , 2007 .

[40]  Francisco Herrera,et al.  A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms , 2011, Swarm Evol. Comput..

[41]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[42]  Fernando José Von Zuben,et al.  The construction of a Boolean competitive neural network using ideas from immunology , 2003, Neurocomputing.

[43]  Jian-Hui Jiang,et al.  Optimized Partition of Minimum Spanning Tree for Piecewise Modeling by Particle Swarm Algorithm. QSAR Studies of Antagonism of Angiotensin II Antagonists , 2004, J. Chem. Inf. Model..

[44]  Gillian Dobbie,et al.  An Evolutionary Particle Swarm Optimization algorithm for data clustering , 2008, 2008 IEEE Swarm Intelligence Symposium.

[45]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[46]  Luís A. Alexandre,et al.  Data classification with multilayer perceptrons using a generalized error function , 2008, Neural Networks.

[47]  Gary G. Yen,et al.  PSO-Based Multiobjective Optimization With Dynamic Population Size and Adaptive Local Archives , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[48]  Leandro Nunes de Castro,et al.  The proposal of a velocity memoryless clustering swarm , 2010, IEEE Congress on Evolutionary Computation.

[49]  A. K. F. Prior,et al.  CPSC: UM ALGORITMO DE ENXAME CONSTRUTIVO PARA AGRUPAMENTO DE DADOS , 2010 .

[50]  E. Wilson,et al.  Sociobiology: The New Synthesis , 1975 .

[51]  B. Aazhang,et al.  An algorithm for training multilayer perceptrons for data classification and function interpolation , 1994 .

[52]  Yiming Yang,et al.  A re-examination of text categorization methods , 1999, SIGIR '99.

[53]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[54]  Bao-Jiang Zhao,et al.  An Ant Colony Clustering Algorithm , 2007, 2007 International Conference on Machine Learning and Cybernetics.

[55]  Thomas Bäck,et al.  Evolutionary computation: Toward a new philosophy of machine intelligence , 1997, Complex..

[56]  S. Kalyani,et al.  Classifier design for static security assessment using particle swarm optimization , 2011, Appl. Soft Comput..

[57]  Xiaohui Cui,et al.  Document Clustering Analysis Based on Hybrid PSO+K-means Algorithm , 2005 .

[58]  Zbigniew Michalewicz,et al.  Evolutionary Computation 1 , 2018 .

[59]  Jonathan Timmis,et al.  Application Areas of AIS: The Past, The Present and The Future , 2005, ICARIS.

[60]  Yanchun Zhang,et al.  Privacy-preserving naive Bayes classification on distributed data via semi-trusted mixers , 2009, Inf. Syst..

[61]  Neil Genzlinger.  A. and Q , 2006 .

[62]  B. Soudan,et al.  An Evolutionary Dynamic Population Size PSO Implementation , 2008, 2008 3rd International Conference on Information and Communication Technologies: From Theory to Applications.

[63]  Andrzej Bargiela,et al.  General fuzzy min-max neural network for clustering and classification , 2000, IEEE Trans. Neural Networks Learn. Syst..

[64]  Guo-Li Shen,et al.  Modified particle swarm optimization algorithm for variable selection in MLR and PLS modeling: QSAR studies of antagonism of angiotensin II antagonists. , 2004, European journal of pharmaceutical sciences : official journal of the European Federation for Pharmaceutical Sciences.

[65]  Leandro N. de Castro,et al.  Data Clustering with Particle Swarms , 2006, 2006 IEEE International Conference on Evolutionary Computation.