Clustering Data by Melting

We derive a new clustering algorithm based on information theory and statistical mechanics, which is the only algorithm that incorporates scale. It also introduces a new concept into clustering: cluster independence. The cluster centers correspond to the local minima of a thermodynamic free energy, which are identified as the fixed points of a one-parameter nonlinear map. The algorithm works by melting the system to produce a tree of clusters in the scale space. Melting is also insensitive to variability in cluster densities, cluster sizes, and ellipsoidal shapes and orientations. We tested the algorithm successfully on both simulated data and a Synthetic Aperture Radar image of an agricultural site with 12 attributes for crop identification.

[1]  Yiu-Fai Wong,et al.  A new clustering algorithm applicable to multispectral and polarimetric SAR images , 1993, IEEE Trans. Geosci. Remote. Sens..

[2]  Isak Gath,et al.  Unsupervised Optimal Fuzzy Clustering , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  John R. Pierce,et al.  Introduction to Communication Science and Systems , 1980 .

[4]  E. Jaynes.  Information Theory and Statistical Mechanics , 1957 .

[5]  Fabio Cocurullo,et al.  A new algorithm for vector quantization , 1995, Proceedings DCC '95 Data Compression Conference.

[6]  Donald J. Newman,et al.  The hexagon theorem , 1982, IEEE Trans. Inf. Theory.

[7]  Orly Yadid-Pecht,et al.  A simple "possibilistic" clustering neural network , 1994, Proceedings of the 12th IAPR International Conference on Pattern Recognition, Vol. 3 - Conference C: Signal Processing (Cat. No.94CH3440-5).

[8]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[9]  Sompolinsky,et al.  Statistical mechanics of the maximum-likelihood density estimation. , 1994, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[10]  J. Wolfe.  PATTERN CLUSTERING BY MULTIVARIATE MIXTURE ANALYSIS. , 1970, Multivariate behavioral research.

[11]  Yiu-Fai Wong,et al.  Scale-space clustering and classification of SAR images with numerous attributes and classes , 1992, [1992] Proceedings IEEE Workshop on Applications of Computer Vision.

[12]  James Kelly,et al.  AutoClass: A Bayesian Classification System , 1993, ML.

[13]  S. Wiggins.  Introduction to Applied Nonlinear Dynamical Systems and Chaos , 1989 .

[14]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[15]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[16]  Anil K. Jain,et al.  Algorithms for Clustering Data , 1988 .

[17]  Geoffrey C. Fox,et al.  A deterministic annealing approach to clustering , 1990, Pattern Recognit. Lett..

[18]  Enrique H. Ruspini,et al.  A New Approach to Clustering , 1969, Inf. Control..

[19]  Dennis Gabor,et al.  Theory of communication , 1946 .

[20]  G H Ball,et al.  A clustering technique for summarizing multivariate data. , 1967, Behavioral science.

[21]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.