On the efficiency of evolutionary fuzzy clustering

Abstract This paper tackles the problem of showing that evolutionary algorithms for fuzzy clustering can be more efficient than systematic (i.e. repetitive) approaches when the number of clusters in a data set is unknown. To do so, a fuzzy version of an Evolutionary Algorithm for Clustering (EAC) is introduced. A fuzzy cluster validity criterion and a fuzzy local search algorithm are used instead of their hard counterparts employed by EAC. Theoretical complexity analyses for both the systematic and evolutionary algorithms under interest are provided. Examples with computational experiments and statistical analyses are also presented.

[1]  Lawrence. Davis,et al.  Handbook Of Genetic Algorithms , 1990 .

[2]  James C. Bezdek,et al.  Generalized fuzzy c-means clustering strategies using Lp norm distances , 2000, IEEE Trans. Fuzzy Syst..

[3]  Ricardo J. G. B. Campello,et al.  A fuzzy extension of the silhouette width criterion for cluster analysis , 2006, Fuzzy Sets Syst..

[4]  Lawrence O. Hall,et al.  Fast Accurate Fuzzy Clustering through Data Reduction , 2003 .

[5]  R. Kruse,et al.  An extension to possibilistic fuzzy cluster analysis , 2004, Fuzzy Sets Syst..

[6]  James C. Bezdek,et al.  A mixed c-means clustering model , 1997, Proceedings of 6th International Fuzzy Systems Conference.

[7]  W. T. Tucker,et al.  Convergence theory for fuzzy c-means: Counterexamples and repairs , 1987, IEEE Transactions on Systems, Man, and Cybernetics.

[8]  James C. Bezdek,et al.  Fuzzy c-means clustering of incomplete data , 2001, IEEE Trans. Syst. Man Cybern. Part B.

[9]  Ricardo J. G. B. Campello,et al.  Evolving clusters in gene-expression data , 2006, Inf. Sci..

[10]  Ricardo J. G. B. Campello,et al.  Evolutionary search for optimal fuzzy c-means clustering , 2004, 2004 IEEE International Conference on Fuzzy Systems (IEEE Cat. No.04CH37542).

[11]  Mukkai S. Krishnamoorthy,et al.  Comparative study of a genetic fuzzy c-means algorithm and a validity guided fuzzy c-means algorithm for locating clusters in noisy data , 1998, 1998 IEEE International Conference on Evolutionary Computation Proceedings. IEEE World Congress on Computational Intelligence (Cat. No.98TH8360).

[12]  M. Kendall.  Elementary Statistics , 1945, Nature.

[13]  Thomas Bäck,et al.  Evolutionary computation: Toward a new philosophy of machine intelligence , 1997, Complex..

[14]  Ricardo J. G. B. Campello,et al.  Evolutionary algorithms for clustering gene-expression data , 2004, Fourth IEEE International Conference on Data Mining (ICDM'04).

[15]  Donald Gustafson,et al.  Fuzzy clustering with a fuzzy covariance matrix , 1978, 1978 IEEE Conference on Decision and Control including the 17th Symposium on Adaptive Processes.

[16]  James M. Keller,et al.  A possibilistic fuzzy c-means clustering algorithm , 2005, IEEE Transactions on Fuzzy Systems.

[17]  Don-Lin Yang,et al.  An efficient Fuzzy C-Means clustering algorithm , 2001, Proceedings 2001 IEEE International Conference on Data Mining.

[18]  James C. Bezdek,et al.  Efficient Implementation of the Fuzzy c-Means Clustering Algorithms , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  M. A. Chapman,et al.  Automated Road Extraction from Satellite Imagery Using Hybrid Genetic Algorithms and Cluster Analysis , 2003 .

[20]  L. Jain,et al.  Fuzzy sets and their application to clustering and training , 2000 .

[21]  Boudewijn P. F. Lelieveldt,et al.  A new cluster validity index for the fuzzy c-mean , 1998, Pattern Recognit. Lett..

[22]  Frank Höppner.  Speeding up fuzzy c-means: using a hierarchical data organisation to control the precision of membership calculation , 2002, Fuzzy Sets Syst..

[23]  G. Klir,et al.  Evolutionary fuzzy c-means clustering algorithm , 1995, Proceedings of 1995 IEEE International Conference on Fuzzy Systems..

[24]  J. Bezdek.  Cluster Validity with Fuzzy Sets , 1973 .

[25]  David B. Fogel,et al.  Evolving fuzzy clusters , 1993, IEEE International Conference on Neural Networks.

[26]  Enrique H. Ruspini,et al.  Numerical methods for fuzzy clustering , 1970, Inf. Sci..

[27]  Mauro Barni,et al.  Comments on "A possibilistic approach to clustering" , 1996, IEEE Trans. Fuzzy Syst..

[28]  James C. Bezdek,et al.  On cluster validity for the fuzzy c-means model , 1995, IEEE Trans. Fuzzy Syst..

[29]  James C. Bezdek,et al.  Complexity reduction for "large image" processing , 2002, IEEE Trans. Syst. Man Cybern. Part B.

[30]  James C. Bezdek,et al.  Relational duals of the c-means clustering algorithms , 1989, Pattern Recognit..

[31]  Frank Klawonn,et al.  Fuzzy clustering with evolutionary algorithms , 1998, Int. J. Intell. Syst..

[32]  Mohamed S. Kamel,et al.  New algorithms for solving the fuzzy clustering problem , 1994, Pattern Recognit..

[33]  Ujjwal Maulik,et al.  Fuzzy partitioning using a real-coded variable-length genetic algorithm for pixel classification , 2003, IEEE Trans. Geosci. Remote. Sens..

[34]  Rajesh N. Davé,et al.  Robust clustering methods: a unified view , 1997, IEEE Trans. Fuzzy Syst..

[35]  David B. Fogel,et al.  Evolutionary Computation: Towards a New Philosophy of Machine Intelligence , 1995 .

[36]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[37]  P. Kersten,et al.  Implementation issues in the fuzzy c-medians clustering algorithm , 1997, Proceedings of 6th International Fuzzy Systems Conference.

[38]  V. J. Rayward-Smith,et al.  Fuzzy Cluster Analysis: Methods for Classification, Data Analysis and Image Recognition , 1999 .

[39]  M. P. Windham.  Cluster validity for fuzzy clustering algorithms , 1981 .

[40]  James E. Gentle,et al.  Finding Groups in Data: An Introduction to Cluster Analysis. , 1991 .

[41]  Mohamed A. Ismail,et al.  Fuzzy clustering for symbolic data , 1998, IEEE Trans. Fuzzy Syst..

[42]  J. Bezdek,et al.  Genetic fuzzy clustering , 1994, NAFIPS/IFIS/NASA '94. Proceedings of the First International Joint Conference of The North American Fuzzy Information Processing Society Biannual Conference. The Industrial Fuzzy Control and Intellige.

[43]  James C. Bezdek,et al.  Some new indexes of cluster validity , 1998, IEEE Trans. Syst. Man Cybern. Part B.

[44]  Sung-Bae Cho,et al.  Evolutionary Fuzzy Clustering Algorithm with Knowledge-Based Evaluation and Applications for Gene Expression Profiling , 2005 .

[45]  Lawrence O. Hall,et al.  Fast fuzzy clustering , 1998, Fuzzy Sets Syst..

[46]  John F. Kolen,et al.  Reducing the time complexity of the fuzzy c-means algorithm , 2002, IEEE Trans. Fuzzy Syst..

[47]  Raghu Krishnapuram,et al.  Fitting an unknown number of lines and planes to image data through compatible cluster merging , 1992, Pattern Recognit..

[48]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[49]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[50]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[51]  James C. Bezdek,et al.  Optimization of fuzzy clustering criteria using genetic algorithms , 1994, Proceedings of the First IEEE Conference on Evolutionary Computation. IEEE World Congress on Computational Intelligence.

[52]  Anil K. Jain,et al.  A Clustering Performance Measure Based on Fuzzy Set Decomposition , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[53]  Mario F. Triola,et al.  Elementary Statistics Using Excel (3rd Edition) , 2006 .

[54]  Lawrence O. Hall,et al.  Scaling genetically guided fuzzy clustering , 1995, Proceedings of 3rd International Symposium on Uncertainty Modeling and Analysis and Annual Conference of the North American Fuzzy Information Processing Society.

[55]  Ricardo J. G. B. Campello,et al.  A Fuzzy Variant of an Evolutionary Algorithm for Clustering , 2007, 2007 IEEE International Fuzzy Systems Conference.

[56]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[57]  James M. Keller,et al.  A possibilistic approach to clustering , 1993, IEEE Trans. Fuzzy Syst..

[58]  James M. Keller,et al.  The possibilistic C-means algorithm: insights and recommendations , 1996, IEEE Trans. Fuzzy Syst..

[59]  Michalis Vazirgiannis,et al.  c ○ 2001 Kluwer Academic Publishers. Manufactured in The Netherlands. On Clustering Validation Techniques , 2022 .

[60]  James C. Bezdek,et al.  Clustering with a genetically optimized approach , 1999, IEEE Trans. Evol. Comput..

[61]  T. Van Le.  Evolutionary fuzzy clustering , 1995, Proceedings of 1995 IEEE International Conference on Evolutionary Computation.

[62]  James C. Bezdek,et al.  Optimization of clustering criteria by reformulation , 1995, IEEE Trans. Fuzzy Syst..

[63]  Jianzhuang Liu,et al.  A genetics-based approach to fuzzy clustering , 1995, Proceedings of 1995 IEEE International Conference on Fuzzy Systems..

[64]  Robert Babuska,et al.  Fuzzy Modeling for Control , 1998 .

[65]  Amanda S. Barnard,et al.  Visualization of Hybridization in Nanocarbon Systems , 2005 .

[66]  M. Narasimha Murty,et al.  Clustering with evolution strategies , 1994, Pattern Recognit..

[67]  Brian Everitt,et al.  Cluster analysis , 1974 .

[68]  Ujjwal Maulik,et al.  A study of some fuzzy cluster validity indices, genetic clustering and application to pixel classification , 2005, Fuzzy Sets Syst..

[69]  Emanuel Falkenauer,et al.  Genetic Algorithms and Grouping Problems , 1998 .

[70]  Ricardo J. G. B. Campello,et al.  Clustering Gene-Expression Data: A Hybrid Approach that Iterates Between k-Means and Evolutionary Search , 2007 .

[71]  James C. Bezdek,et al.  Optimal Fuzzy Partitions: A Heuristic for Estimating the Parameters in a Mixture of Normal Distributions , 1975, IEEE Transactions on Computers.

[72]  Gerardo Beni,et al.  A Validity Measure for Fuzzy Clustering , 1991, IEEE Trans. Pattern Anal. Mach. Intell..

[73]  James C. Bezdek,et al.  Nerf c-means: Non-Euclidean relational fuzzy clustering , 1994, Pattern Recognit..

[74]  R. Howard,et al.  Local convergence analysis of a grouped variable version of coordinate descent , 1987 .

[75]  Nelson F. F. Ebecken,et al.  A genetic algorithm for cluster analysis , 2003, Intell. Data Anal..

[76]  R. Krishnapuram,et al.  A fuzzy relative of the k-medoids algorithm with application to web document and snippet clustering , 1999, FUZZ-IEEE'99. 1999 IEEE International Fuzzy Systems. Conference Proceedings (Cat. No.99CH36315).

[77]  Uzay Kaymak,et al.  Fuzzy clustering with volume prototypes and adaptive cluster merging , 2002, IEEE Trans. Fuzzy Syst..

[78]  Ujjwal Maulik,et al.  Genetic clustering for automatic evolution of clusters and application to image classification , 2002, Pattern Recognit..