The Local Minima Problem in Hierarchical Classes Analysis: An Evaluation of a Simulated Annealing Algorithm and Various Multistart Procedures

Abstract Hierarchical classes models are quasi-order retaining Boolean decomposition models for N-way N-mode binary data. To fit these models to data, rationally started alternating least squares (or, equivalently, alternating least absolute deviations) algorithms have been proposed. Extensive simulation studies showed that these algorithms succeed quite well in recovering the underlying truth but frequently end in a local minimum. In this paper we evaluate whether or not this local minimum problem can be mitigated by means of two common strategies for avoiding local minima in combinatorial data analysis: simulated annealing (SA) and use of a multistart procedure. In particular, we propose a generic SA algorithm for hierarchical classes analysis and three different types of random starts. The effectiveness of the SA algorithm and the random starts is evaluated by reanalyzing data sets of previous simulation studies. The reported results support the use of the proposed SA algorithm in combination with a random multistart procedure, regardless of the properties of the data set under study.

[1]  Willem J. Heiser,et al.  A Permutation-Translation Simulated Annealing Algorithm for L1 and L2 Unidimensional Scaling , 2005, J. Classif..

[2]  Iven Van Mechelen,et al.  Individual differences in anger and sadness: in pursuit of active situational features and psychological processes. , 2006, Journal of personality.

[3]  Khaled S. Al-Sultan,et al.  Computational experience on four algorithms for the hard clustering problem , 1996, Pattern Recognit. Lett..

[4]  Iven Van Mechelen,et al.  Individual Differences in Situation-Behavior Profiles : A Triple Typology Model , 2001 .

[5]  P. Boeck,et al.  Hierarchical classes: Model and data analysis , 1988 .

[6]  Iven Van Mechelen,et al.  A Branch-and-bound Algorithm for Boolean Regression , 1998 .

[7]  J. K. Lenstra,et al.  Local Search in Combinatorial Optimisation. , 1997 .

[8]  Eva Ceulemans,et al.  Individual differences in patterns of appraisal and anger experience , 2007 .

[9]  Richard C. Dubes,et al.  Experiments in projection and clustering by simulated annealing , 1989, Pattern Recognit..

[10]  Iven Van Mechelen,et al.  Hierarchical classes models for three-way three-mode binary data: interrelations and model selection , 2005 .

[11]  Iven Van Mechelen,et al.  An Evaluation of Two Algorithms for Hierarchical Classes Analysis , 2001, J. Classif..

[12]  Douglas Steinley,et al.  Local optima in K-means clustering: what you don't know may hurt you. , 2003, Psychological methods.

[13]  G. W. Milligan,et al.  An examination of the effect of six types of error perturbation on fifteen clustering algorithms , 1980 .

[14]  M. Gara.  A Set-Theoretical Model of Person Perception. , 1990, Multivariate behavioral research.

[15]  Shokri Z. Selim,et al.  K-Means-Type Algorithms: A Generalized Convergence Theorem and Characterization of Local Optimality , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Iven Van Mechelen,et al.  The conjunctive model of hierarchical classes , 1995 .

[17]  I. Mechelen,et al.  Implicit Taxonomy in Psychiatric Diagnosis: A Case Study , 1989 .

[18]  Iven Van Mechelen,et al.  Tucker3 hierarchical classes analysis , 2003 .

[19]  I. Vanmechelen,et al.  A Latent Criteria Model for Choice Data , 1994 .

[20]  David J. Hand,et al.  Short communication: Optimising k-means clustering results with standard software packages , 2005 .

[21]  Michael J. Brusco,et al.  Using Quadratic Assignment Methods to Generate Initial Permutations for Least-Squares Unidimensional Scaling of Symmetric Proximity Matrices , 2000, J. Classif..

[22]  Iven Van Mechelen,et al.  Indclas: A three-way hierarchical classes model , 1999 .

[23]  Michael J. Brusco.  A Simulated Annealing Heuristic for Unidimensional and Multidimensional (City-Block) Scaling of Symmetric Proximity Matrices , 2001, J. Classif..

[24]  Iven Van Mechelen,et al.  Tucker2 hierarchical classes analysis , 2004 .