An Empirical Evaluation of Different Initializations on the Number of K-Means Iterations

This paper presents an analysis of the number of iterations K-Means takes to converge under different initializations. We have experimented with seven initialization algorithms in a total of 37 real and synthetic datasets. We have found that hierarchical-based initializations tend to be most effective at reducing the number of iterations, especially a divisive algorithm using the Ward criterion when applied to real datasets.

[1]  G. W. Milligan,et al.  The validation of four ultrametric clustering algorithms , 1980, Pattern Recognit..

[2]  Renato Cordeiro de Amorim,et al.  Minkowski metric, feature weighting and anomalous cluster initializing in K-Means clustering , 2012, Pattern Recognit..

[3]  Sankar K. Pal,et al.  Pattern Recognition Algorithms for Data Mining: Scalability, Knowledge Discovery, and Soft Granular Computing , 2004 .

[4]  Boris Mirkin,et al.  Mathematical Classification and Clustering , 1996 .

[5]  D. M. Hutton,et al.  Pattern Recognition Algorithms for Data Mining , 2005 .

[6]  David G. Stork,et al.  Pattern Classification , 1973 .

[7]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[8]  David G. Stork,et al.  Pattern Classification (2nd ed.) , 1999 .

[9]  Boris Mirkin,et al.  Clustering For Data Mining: A Data Recovery Approach (Chapman & Hall/Crc Computer Science) , 2005 .

[10]  Anil K. Jain.  Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[11]  J. H. Ward.  Hierarchical Grouping to Optimize an Objective Function , 1963 .

[12]  Anil K. Jain.  Data clustering: 50 years beyond K-means , 2010, Pattern Recognit. Lett..

[13]  Ann Q. Gates,et al.  TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING , 2005 .

[14]  Ali S. Hadi,et al.  Finding Groups in Data: An Introduction to Chster Analysis , 1991 .

[15]  Peter Komisarczuk,et al.  On Initializations for the Minkowski Weighted K-Means , 2012, IDA.

[16]  G H Ball,et al.  A clustering technique for summarizing multivariate data. , 1967, Behavioral science.

[17]  Pedro Larrañaga,et al.  Applying genetic algorithms to search for the best hierarchical clustering of a dataset , 1999, Pattern Recognit. Lett..

[18]  Frank Klawonn,et al.  Advances in Intelligent Data Analysis XI , 2012, Lecture Notes in Computer Science.

[19]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[20]  Pedro Larrañaga,et al.  An empirical comparison of four initialization methods for the K-Means algorithm , 1999, Pattern Recognit. Lett..

[21]  R. C. de Amorim,et al.  Constrained Intelligent K-Means: Improving Results with Limited Previous Knowledge. , 2008 .

[22]  Shigeo Abe DrEng.  Pattern Classification , 2001, Springer London.

[23]  G. W. Milligan,et al.  A study of standardization of variables in cluster analysis , 1988 .

[24]  Boris G. Mirkin,et al.  Intelligent Choice of the Number of Clusters in K-Means Clustering: An Experimental Study with Different Cluster Spreads , 2010, J. Classif..

[25]  Sergei Vassilvitskii,et al.  k-means++: the advantages of careful seeding , 2007, SODA '07.

[26]  Robert Henson,et al.  OCLUS: An Analytic Method for Generating Clusters with Known Overlap , 2005, J. Classif..

[27]  Michael J. Brusco,et al.  Initializing K-means Batch Clustering: A Critical Evaluation of Several Techniques , 2007, J. Classif..