K-Means-Type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality

The K-means algorithm is a commonly used technique in cluster analysis. In this paper, several questions about the algorithm are addressed. The clustering problem is first cast as a nonconvex mathematical program. Then, a rigorous proof of the finite convergence of the K-means-type algorithm is given for any metric. It is shown that under certain conditions the algorithm may fail to converge to a local minimum, and that it converges under differentiability conditions to a Kuhn-Tucker point. Finally, a method for obtaining a local-minimum solution is given.

[1]  Leon S. Lasdon,et al.  Optimization Theory of Large Systems , 1970 .

[2]  G. W. Milligan,et al.  An examination of the effect of six types of error perturbation on fifteen clustering algorithms , 1980 .

[3]  Julius T. Tou,et al.  Pattern Recognition Principles , 1974 .

[4]  Brian Everitt,et al.  Cluster analysis , 1974 .

[5]  Richard E. Wendell,et al.  Minimization of a Non-Separable Objective Function Subject to Disjoint Constraints , 1976, Oper. Res..

[6]  Geoffrey H. Ball,et al.  ISODATA, A NOVEL METHOD OF DATA ANALYSIS AND PATTERN CLASSIFICATION , 1965 .

[7]  Anil K. Jain,et al.  Clustering Methodologies in Exploratory Data Analysis , 1980, Adv. Comput..

[8]  J. Mezzich.  Evaluating clustering methods for psychiatric diagnosis. , 1978, Biological psychiatry.

[9]  James C. Bezdek,et al.  A Convergence Theorem for the Fuzzy ISODATA Clustering Algorithms , 1980, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Anil K. Jain,et al.  Clustering techniques: The user's dilemma , 1976, Pattern Recognit..

[11]  E. Filsinger,et al.  An empirical typology of adjustment to aging. , 1978, Journal of gerontology.

[12]  Michael R. Anderberg,et al.  Cluster Analysis for Applications , 1973 .

[13]  James C. Bezdek,et al.  Fuzzy mathematics in pattern classification , 1973 .

[14]  Richard E. Wendell,et al.  Technical Note - Further Results on the Minimization of a Nonseparable Objective Function Subject to Disjoint Constraints , 1980, Oper. Res..

[15]  Leon Cooper,et al.  N‐DIMENSIONAL LOCATION MODELS: AN APPLICATION TO CLUSTER ANALYSIS , 1973 .

[16]  E. Forgy.  Cluster analysis of multivariate data : efficiency versus interpretability of classifications , 1965 .

[17]  Richard O. Duda,et al.  Pattern classification and scene analysis , 1974, A Wiley-Interscience publication.

[18]  Enrique H. Ruspini.  New experimental results in fuzzy clustering , 1973, Inf. Sci..

[19]  Mezzich Je.  Evaluating clustering methods for psychiatric diagnosis. , 1978 .

[20]  R. Jancey.  Multidimensional group analysis , 1966 .

[21]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .