Cluster differences scaling with a within-clusters loss component and a fuzzy successive approximation strategy to avoid local minima

Cluster differences scaling is a method for partitioning a set of objects into classes and simultaneously finding a low-dimensional spatial representation ofK cluster points, to model a given square table of dissimilarities amongn stimuli or objects. The least squares loss function of cluster differences scaling, originally defined only on the residuals of pairs of objects that are allocated to different clusters, is extended with a loss component for pairs that are allocated to the same cluster. It is shown that this extension makes the method equivalent to multidimensional scaling with cluster constraints on the coordinates. A decomposition of the sum of squared dissimilarities into contributions from several sources of variation is described, including the appropriate degrees of freedom for each source. After developing a convergent algorithm for fitting the cluster differences model, it is argued that the individual objects and the cluster locations can be jointly displayed in a configuration obtained as a by-product of the optimization. Finally, the paper introduces a fuzzy version of the loss function, which can be used in a successive approximation strategy for avoiding local minima. A simulation study demonstrates that this strategy significantly outperforms two other well-known initialization strategies, and that it has a success rate of 92 out of 100 in attaining the global minimum.

[1]  R. Fisher THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[2]  Walter D. Fisher On Grouping for Maximum Homogeneity , 1958 .

[3]  Robert R. Sokal,et al.  A statistical method for evaluating systematic relationships , 1958 .

[4]  R. Shepard The analysis of proximities: Multidimensional scaling with an unknown distance function. II , 1962 .

[5]  Calyampudi Radhakrishna Rao,et al.  Linear Statistical Inference and its Applications , 1967 .

[6]  N. Draper,et al.  Applied Regression Analysis , 1966 .

[7]  G H Ball,et al.  A clustering technique for summarizing multivariate data. , 1967, Behavioral science.

[8]  L. Guttman A general nonmetric technique for finding the smallest coordinate space for a configuration of points , 1968 .

[9]  Enrique H. Ruspini,et al.  Numerical methods for fuzzy clustering , 1970, Inf. Sci..

[10]  Brian W. Kernighan,et al.  An efficient heuristic procedure for partitioning graphs , 1970, Bell Syst. Tech. J..

[11]  James C. Bezdek,et al.  Fuzzy mathematics in pattern classification , 1973 .

[12]  J. C. Dunn,et al.  A Fuzzy Relative of the ISODATA Process and Its Use in Detecting Compact Well-Separated Clusters , 1973 .

[13]  Calyampudi R. Rao,et al.  Linear Statistical Inference and Its Applications. , 1975 .

[14]  James C. Bezdek,et al.  Optimal Fuzzy Partitions: A Heuristic for Estimating the Parameters in a Mixture of Normal Distributions , 1975, IEEE Transactions on Computers.

[15]  Waldo R. Tobler,et al.  Spatial Interaction Patterns , 1976 .

[16]  C. F. Banfield,et al.  Algorithm AS 113: A Transfer for Non-Hierarchical Classification , 1977 .

[17]  J. Kruskal The Relationship between Multidimensional Scaling and Clustering , 1977 .

[18]  A. D. Gordon,et al.  An Algorithm for Euclidean Sum of Squares Classification , 1977 .

[19]  J. A. Hartigan,et al.  A k-means clustering algorithm , 1979 .

[20]  Maurice K. Wong,et al.  Algorithm AS136: A k-means clustering algorithm. , 1979 .

[21]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[22]  A. D. Gordon,et al.  Classification : Methods for the Exploratory Analysis of Multivariate Data , 1981 .

[23]  Shokri Z. Selim,et al.  K-Means-Type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  D. Hand Cluster dissection and analysis: Helmuth SPATH Wiley, Chichester, 1985, 226 pages, £25.00 , 1986 .

[25]  Hans-Hermann Bock,et al.  On the Interface between Cluster Analysis, Principal Component Analysis, and Multidimensional Scaling , 1987 .

[26]  Generalised canonical analysis. , 1989 .

[27]  Michel Wedel,et al.  Clusterwise regression and market segmentation : developments and applications , 1990 .

[28]  Patrick J. F. Groenen,et al.  The majorization approach to multidimensional scaling : some problems and extensions , 1993 .

[29]  Willem J. Heiser,et al.  Clustering in Low-Dimensional Space , 1993 .

[30]  L. Zadeh Fuzzy sets and their application to pattern classification and clustering analysis , 1996 .