Dynamic clustering using particle swarm optimization with application in image segmentation

A new dynamic clustering approach (DCPSO), based on particle swarm optimization, is proposed. This approach is applied to image segmentation. The proposed approach automatically determines the “optimum” number of clusters and simultaneously clusters the data set with minimal user interference. The algorithm starts by partitioning the data set into a relatively large number of clusters to reduce the effects of initial conditions. Using binary particle swarm optimization the “best” number of clusters is selected. The centers of the chosen clusters is then refined via the K-means clustering algorithm. The proposed approach was applied on both synthetic and natural images. The experiments conducted show that the proposed approach generally found the “optimum” number of clusters on the tested images. A genetic algorithm and random search version of dynamic clustering is presented and compared to the particle swarm version.

[1]  Hazem M. Abbas,et al.  Neural networks for maximum likelihood clustering , 1994, Signal Process..

[2]  Siddheswar Ray,et al.  Determination of Number of Clusters in K-Means Clustering and Application in Colour Image Segmentation , 2000 .

[3]  Frans van den Bergh,et al.  An analysis of particle swarm optimizers , 2002 .

[4]  Andries P. Engelbrecht,et al.  Computational Intelligence: An Introduction , 2002 .

[5]  BischofHorst,et al.  MDL Principle for Robust Vector Quantisation , 1999 .

[6]  Yeuvo Jphonen,et al.  Self-Organizing Maps , 1995 .

[7]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[8]  Yue Shi,et al.  A modified particle swarm optimizer , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[9]  James Kennedy,et al.  Small worlds and mega-minds: effects of neighborhood topology on particle swarm performance , 1999, Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406).

[10]  Nozha Boujemaa.  On competitive unsupervised clustering , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[11]  A. Engelbrecht,et al.  A new locally convergent particle swarm optimiser , 2002, IEEE International Conference on Systems, Man and Cybernetics.

[12]  David L. Dowe,et al.  Intrinsic classification by MML - the Snob program , 1994 .

[13]  Claudio Carpineto,et al.  A lattice conceptual clustering system and its application to browsing retrieval , 2004, Machine Learning.

[14]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[15]  Suganthan.  [IEEE 1999. Congress on Evolutionary Computation-CEC99 - Washington, DC, USA (6-9 July 1999)] Proceedings of the 1999 Congress on Evolutionary Computation-CEC99 (Cat. No. 99TH8406) - Particle swarm optimiser with neighbourhood operator , 1999 .

[16]  G. McLachlan,et al.  The EM algorithm and extensions , 1996 .

[17]  K. Huang,et al.  A synergistic automatic clustering technique (SYNERACT) for multispectral image Analysis , 2002 .

[18]  Hichem Frigui,et al.  A Robust Competitive Clustering Algorithm With Applications in Computer Vision , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  M. Vazirgiannis,et al.  Clustering validity assessment using multi representatives , 2002 .

[20]  Anil K. Jain,et al.  Large-Scale Parallel Data Clustering , 1998, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Anil K. Jain,et al.  Data clustering: a review , 1999, CSUR.

[22]  Josiane Zerubia,et al.  Fully unsupervised fuzzy clustering with entropy criterion , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[23]  Donald W. Bouldin,et al.  A Cluster Separation Measure , 1979, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  James C. Bezdek,et al.  A Convergence Theorem for the Fuzzy ISODATA Clustering Algorithms , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  C. S. Wallace,et al.  An Information Measure for Classification , 1968, Comput. J..

[26]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[27]  J. Dunn.  Well-Separated Clusters and Optimal Fuzzy Partitions , 1974 .

[28]  Stephen M. Smith,et al.  Segmentation of brain MR images through a hidden Markov random field model and the expectation-maximization algorithm , 2001, IEEE Transactions on Medical Imaging.

[29]  Michalis Vazirgiannis,et al.  On Clustering Validation Techniques , 2001, Journal of Intelligent Information Systems.

[30]  Anil K. Jain,et al.  Statistical Pattern Recognition: A Review , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  R. Redner,et al.  Mixture densities, maximum likelihood, and the EM algorithm , 1984 .

[32]  Andries P. Engelbrecht,et al.  SIGT: SYNTHETIC IMAGE GENERATION TOOL FOR CLUSTERING ALGORITHMS , 2005 .

[33]  Gregory James Hamerly,et al.  Learning structure and concepts in data through data clustering , 2003 .

[34]  Sankar K. Pal,et al.  A review on image segmentation techniques , 1993, Pattern Recognit..

[35]  J. Kennedy,et al.  Population structure and particle swarm performance , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[36]  L. Wasserman,et al.  A Reference Bayesian Test for Nested Hypotheses and its Relationship to the Schwarz Criterion , 1995 .

[37]  King-Sun Fu,et al.  A survey on image segmentation , 1981, Pattern Recognit..

[38]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Abhijit S. Pandya,et al.  Pattern Recognition with Neural Networks in C++ , 1995 .

[40]  Kishan G. Mehrotra,et al.  Elements of artificial neural networks , 1996 .

[41]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[42]  E. R. Davies,et al.  Machine vision - theory, algorithms, practicalities , 2004 .

[43]  Russell C. Eberhart,et al.  A discrete binary version of the particle swarm algorithm , 1997, 1997 IEEE International Conference on Systems, Man, and Cybernetics. Computational Cybernetics and Simulation.

[44]  Greg Hamerly,et al.  Learning the k in k-means , 2003, NIPS.

[45]  Erik K. Antonsson,et al.  Dynamic partitional clustering using evolution strategies , 2000, 2000 26th Annual Conference of the IEEE Industrial Electronics Society. IECON 2000. 2000 IEEE International Conference on Industrial Electronics, Control and Instrumentation. 21st Century Technologies.

[46]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[47]  E. Forgy,et al.  Cluster analysis of multivariate data : efficiency versus interpretability of classifications , 1965 .

[48]  G.B. Coleman,et al.  Image segmentation by clustering , 1979, Proceedings of the IEEE.

[49]  Umeshwar Dayal,et al.  K-Harmonic Means - A Data Clustering Algorithm , 1999 .

[50]  James Kennedy,et al.  Particle swarm optimization , 2002, Proceedings of ICNN'95 - International Conference on Neural Networks.

[51]  Russell C. Eberhart,et al.  Parameter Selection in Particle Swarm Optimization , 1998, Evolutionary Programming.

[52]  Sid Ray,et al.  Clustering-based colour image segmentation using inter-cluster distance , 1997 .

[53]  Bin Zhang.  Generalized K-Harmonic Means -- Boosting in Unsupervised Learning , 2000 .

[54]  Sergios Theodoridis,et al.  Pattern Recognition, Third Edition , 2006 .

[55]  Mauro Birattari,et al.  Swarm Intelligence , 2012, Lecture Notes in Computer Science.

[56]  Christophe Rosenberger,et al.  Unsupervised clustering method with optimal estimation of the number of clusters: application to image segmentation , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[57]  Andrew W. Moore,et al.  X-means: Extending K-means with Efficient Estimation of the Number of Clusters , 2000, ICML.

[58]  James C. Bezdek,et al.  Nearest prototype classification: clustering, genetic algorithms, or random search? , 1998, IEEE Trans. Syst. Man Cybern. Part C.

[59]  Bin Zhang,et al.  Genera lized K- Harmonic Means - - Boosting in Unsupervised Learnin g , 2000 .

[60]  Yee Leung,et al.  Clustering by Scale-Space Filtering , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[61]  G H Ball,et al.  A clustering technique for summarizing multivariate data. , 1967, Behavioral science.

[62]  Andries Petrus Engelbrecht,et al.  Particle swarm optimization method for image clustering , 2005, Int. J. Pattern Recognit. Artif. Intell..

[63]  Hichem Frigui,et al.  Clustering by competitive agglomeration , 1997, Pattern Recognit..

[64]  Michalis Vazirgiannis,et al.  Clustering validity assessment: finding the optimal partitioning of a data set , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[65]  Jonathan J. Oliver.  Introduction to Minimum Encoding Inference , 1994 .