Multiple Cooperating Swarms for Data Clustering

A new clustering technique by the use of multiple swarms is proposed. The proposed technique mimics the behavior of biological swarms which explore food situated in several places. We model the clustering problem using particle swarm optimization (PSO) approach. The proposed method considers multiple cooperating swarms to find centers of clusters. By assigning a portion of the solution space to each swarm, the exploration ability to find the solution is enhanced. Moreover, the cooperation among swarms increases the between-class distance. The proposed method outperforms k-means clustering as well as conventional PSO-based clustering techniques

[1]  Russell C. Eberhart,et al.  Multiobjective optimization using dynamic neighborhood particle swarm optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[2]  Mary P. Harper,et al.  Introducing Speech and Language Processing, by John Coleman , 2005, CL.

[3]  Thomas E. Potok,et al.  Document clustering using particle swarm optimization , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[4]  Fakhri Karray,et al.  Particle swarm clustering ensemble , 2008, GECCO '08.

[5]  Fakhri Karray,et al.  Hybrid learning scheme for modular-based phoneme recognizer , 2007, 2007 9th International Symposium on Signal Processing and Its Applications.

[6]  Robert J. Schalkoff,et al.  Pattern recognition - statistical, structural and neural approaches , 1991 .

[7]  Anthony J. Robinson,et al.  An application of recurrent nets to phone probability estimation , 1994, IEEE Trans. Neural Networks.

[8]  John Coleman,et al.  Introducing Speech and Language Processing (Cambridge Introductions to Language and Linguistics) , 2005 .

[9]  R.S. Bajwa,et al.  Simultaneous speech segmentation and phoneme recognition using dynamic programming , 1996, 1996 IEEE International Conference on Acoustics, Speech, and Signal Processing Conference Proceedings.

[10]  T. D. Harrison,et al.  A connectionist model for phoneme recognition in continuous speech , 1989, International Conference on Acoustics, Speech, and Signal Processing,.

[11]  Ching-Yi Chen,et al.  Alternative KPSO-Clustering Algorithm , 2005 .

[12]  Fuad M. Alkoot.  Design of multiple classifier systems , 2001 .

[13]  Andries P. Engelbrecht,et al.  Training support vector machines with particle swarms , 2003, Proceedings of the International Joint Conference on Neural Networks, 2003..

[14]  John H. L. Hansen,et al.  Discrete-Time Processing of Speech Signals , 1993 .

[15]  Thomas E. Potok,et al.  A flocking based algorithm for document clustering analysis , 2006, J. Syst. Archit..

[16]  Michael J. Panik.  Advanced Statistics from an Elementary Point of View , 2005 .

[17]  Fakhri Karray,et al.  Cooperative Swarms for Clustering Phoneme Data , 2007, 2007 IEEE/SP 14th Workshop on Statistical Signal Processing.

[18]  Petra Perner,et al.  Data Mining - Concepts and Techniques , 2002, Künstliche Intell..

[19]  Yasuo Ariki,et al.  Hierarchical phoneme recognition by hidden Markov models based on multiple feature integration , 1989 .

[20]  Robert A. Jacobs,et al.  Hierarchical Mixtures of Experts and the EM Algorithm , 1993, Neural Computation.

[21]  K. Aikawa,et al.  Speech recognition using time-warping neural networks , 1991, Neural Networks for Signal Processing Proceedings of the 1991 IEEE Workshop.

[22]  Caroline Barrière,et al.  Fast two-level-dynamic-programming algorithm for speech recognition , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[23]  Alex Alves Freitas,et al.  A hybrid particle swarm/ant colony algorithm for the classification of hierarchical biological data , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[24]  M. N. Vrahatis,et al.  Particle swarm optimization method in multiobjective problems , 2002, SAC '02.

[25]  William M. Schaffer,et al.  Competition for nectar between introduced honey bees and native North American bees and ants. , 1983 .

[26]  Riccardo Poli,et al.  Particle swarm optimization , 1995, Swarm Intelligence.

[27]  Andries Petrus Engelbrecht,et al.  Fundamentals of Computational Swarm Intelligence , 2005 .

[28]  Michael J. Watts,et al.  Simple evolving connectionist systems and experiments on isolated phoneme recognition , 2000, 2000 IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks. Proceedings of the First IEEE Symposium on Combinations of Evolutionary Computation and Neural Networks (Cat. No.00.

[29]  Baldo Faieta,et al.  Diversity and adaptation in populations of clustering ants , 1994 .

[30]  John Fulcher,et al.  Computational Intelligence: An Introduction , 2008, Computational Intelligence: A Compendium.

[31]  Antony Browne,et al.  Multistage Neural Network Ensembles , 2002, Multiple Classifier Systems.

[32]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[33]  Joseph Picone,et al.  Applications of support vector machines to speech recognition , 2004, IEEE Transactions on Signal Processing.

[34]  Michalis Vazirgiannis,et al.  Clustering validity assessment: finding the optimal partitioning of a data set , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[35]  M.G. Bellanger,et al.  Digital processing of speech signals , 1980, Proceedings of the IEEE.

[36]  K. Aikawa.  Time-warping neural network for phoneme recognition , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[37]  Fakhri Karray,et al.  Model order selection for multiple cooperative swarms clustering using stability analysis , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[38]  Rolf Drechsler,et al.  Applications of Evolutionary Computing, EvoWorkshops 2008: EvoCOMNET, EvoFIN, EvoHOT, EvoIASP, EvoMUSART, EvoNUM, EvoSTOC, and EvoTransLog, Naples, Italy, March 26-28, 2008. Proceedings , 2008, EvoWorkshops.

[39]  Nicolas Monmarché,et al.  On Improving Clustering in Numerical Databases with Artificial Ants , 1999, ECAL.

[40]  Andries Petrus Engelbrecht,et al.  A Cooperative approach to particle swarm optimization , 2004, IEEE Transactions on Evolutionary Computation.

[41]  Russell C. Eberhart,et al.  Gene clustering using self-organizing maps and particle swarm optimization , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[42]  Jean-Louis Deneubourg,et al.  The dynamics of collective sorting robot-like ants and ant-like robots , 1991 .

[43]  Russell C. Eberhart,et al.  Particle swarm with extended memory for multiobjective optimization , 2003, Proceedings of the 2003 IEEE Swarm Intelligence Symposium. SIS'03 (Cat. No.03EX706).

[44]  Abdelkader Benyettou,et al.  Continuous speech recognition by adaptive temporal radial basis function , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[45]  Steven E. Golowich,et al.  A Support Vector/Hidden Markov Model Approach to Phoneme Recognition , 1998 .

[46]  Umeshwar Dayal,et al.  K-Harmonic Means - A Data Clustering Algorithm , 1999 .

[47]  Richard C. Chapman,et al.  Application of Particle Swarm to Multiobjective Optimization , 1999 .

[48]  Geoffrey E. Hinton,et al.  Adaptive Mixtures of Local Experts , 1991, Neural Computation.

[49]  Fakhri Karray,et al.  Soft Computing and Tools of Intelligent Systems Design: Theory and Applications , 2004 .

[50]  Joachim M. Buhmann,et al.  Stability-Based Validation of Clustering Solutions , 2004, Neural Computation.

[51]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[52]  Mohammed El-Abd,et al.  Information exchange in multiple cooperating swarms , 2005, Proceedings 2005 IEEE Swarm Intelligence Symposium, 2005. SIS 2005..

[53]  Ludmila I. Kuncheva,et al.  Evaluation of Stability of k-Means Cluster Ensembles with Respect to Random Initialization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  T. J. Reynolds,et al.  Phoneme classification with multinets , 1998, ICSP '98. 1998 Fourth International Conference on Signal Processing (Cat. No.98TH8344).

[55]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[56]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[57]  Radford M. Neal.  Pattern Recognition and Machine Learning , 2007, Technometrics.

[58]  Masami Ito,et al.  Task decomposition and module combination based on class relations: a modular neural network for pattern classification , 1999, IEEE Trans. Neural Networks.

[59]  Andries Petrus Engelbrecht,et al.  Dynamic clustering using particle swarm optimization with application in image segmentation , 2006, Pattern Analysis and Applications.

[60]  Caro Lucas,et al.  Swarm Clustering Based on Flowers Pollination by Artificial Bees , 2006, Swarm Intelligence in Data Mining.

[61]  Alex Waibel,et al.  Phoneme recognition: neural networks vs. hidden Markov models vs. hidden Markov models , 1988, ICASSP-88., International Conference on Acoustics, Speech, and Signal Processing.

[62]  A. Glaeser.  Compact modular neural networks in a hybrid speaker-independent speech recognition system , 1996, Proceedings of International Conference on Neural Networks (ICNN'96).

[63]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[64]  H. Tomabechi,et al.  Phoneme recognition using a time-sliced recurrent recognizer , 1994, Proceedings of 1994 IEEE International Conference on Neural Networks (ICNN'94).

[65]  Mohamed S. Kamel,et al.  Modular neural networks: a survey. , 1999, International journal of neural systems.

[66]  Fakhreddine O. Karray,et al.  Soft Computing and Intelligent Systems Design, Theory, Tools and Applications , 2006, IEEE Transactions on Neural Networks.

[67]  Andries Petrus Engelbrecht,et al.  Data clustering using particle swarm optimization , 2003, The 2003 Congress on Evolutionary Computation, 2003. CEC '03..

[68]  Michalis Vazirgiannis,et al.  On Clustering Validation Techniques , 2001, Journal of Intelligent Information Systems.

[69]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[70]  Geoffrey E. Hinton,et al.  Phoneme recognition using time-delay neural networks , 1989, IEEE Trans. Acoust. Speech Signal Process..

[71]  Lawrence K. Saul,et al.  Large Margin Gaussian Mixture Modeling for Phonetic Classification and Recognition , 2006, 2006 IEEE International Conference on Acoustics Speech and Signal Processing Proceedings.

[72]  Andries Petrus Engelbrecht,et al.  Particle swarm optimization method for image clustering , 2005, Int. J. Pattern Recognit. Artif. Intell..

[73]  Parag M. Kanade,et al.  Fuzzy ants as a clustering concept , 2003, 22nd International Conference of the North American Fuzzy Information Processing Society, NAFIPS 2003.

[74]  David G. Stork,et al.  Pattern Classification , 1973 .

[75]  F. Karray,et al.  Modular-Based Classifier for Phoneme Recognition , 2006, 2006 IEEE International Symposium on Signal Processing and Information Technology.

[76]  Shigeki Sagayama,et al.  A pairwise discriminant approach to robust phoneme recognition by time-delay neural networks , 1991, [Proceedings] ICASSP 91: 1991 International Conference on Acoustics, Speech, and Signal Processing.

[77]  Russell C. Eberhart,et al.  A new optimizer using particle swarm theory , 1995, MHS'95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science.

[78]  C.A. Coello Coello,et al.  MOPSO: a proposal for multiple objective particle swarm optimization , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).