Global Optimization in Any Minkowski Metric: A Permutation-Translation Simulated Annealing Algorithm for Multidimensional Scaling

It is well known that considering a non-Euclidean Minkowski metric in Multidimensional Scaling, either for the distance model or for the loss function, increases the computational problem of local minima considerably. In this paper, we propose an algorithm in which both the loss function and the composition rule can be considered in any Minkowski metric, using a multivariate randomly alternating Simulated Annealing procedure with permutation and translation phases. The algorithm has been implemented in Fortran and tested over classical and simulated data matrices with sizes up to 200 objects. A study has been carried out with some of the common loss functions to determine the most suitable values for the main parameters. The experimental results confirm the theoretical expectation that Simulated Annealing is a suitable strategy to deal by itself with the optimization problems in Multidimensional Scaling, in particular for City-Block, Euclidean and Infinity metrics.

[1]  Willem J. Heiser,et al.  A Permutation-Translation Simulated Annealing Algorithm for L1 and L2 Unidimensional Scaling , 2005, J. Classif..

[2]  I. Borg.  Multidimensional similarity structure analysis , 1987 .

[3]  R. Shepard.  Representation of structure in similarity data: Problems and prospects , 1974 .

[4]  J. O. De Meira Penna,et al.  A psychological approach to ethics , 1985, Journal of Religion and Health.

[5]  P. Groenen,et al.  The tunneling method for global optimization in multidimensional scaling , 1996 .

[6]  Phipps Arabie,et al.  Was euclid an unnecessarily sophisticated psychologist? , 1991 .

[7]  Ian Barrodale,et al.  Algorithm 552: Solution of the Constrained I1 Linear Approximation Problem [F4] , 1980, TOMS.

[8]  G. De Soete,et al.  On the use of simulated annealing for combinatorial data analysis , 1988 .

[9]  Patrick J. F. Groenen,et al.  The majorization approach to multidimensional scaling : some problems and extensions , 1993 .

[10]  L. Hubert,et al.  Multidimensional scaling in the city-block metric: A combinatorial approach , 1992 .

[11]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[12]  P. Groenen,et al.  The majorization approach to multidimensional scaling for Minkowski distances , 1995 .

[13]  J. Kruskal.  Multidimensional scaling by optimizing goodness of fit to a nonmetric hypothesis , 1964 .

[14]  W. R. Garner.  Uncertainty and structure as psychological concepts , 1975 .

[15]  Forrest W. Young,et al.  Nonmetric individual differences multidimensional scaling: An alternating least squares method with optimal scaling features , 1977 .

[16]  W. Fischer,et al.  More about metrics of subjective spaces and attention distributions , 1972 .

[17]  Sandra G. Funk.  The Perceived Structure of American Ethnic Groups: The Use of Multidimensional Scaling in Stereotype Research , 1976 .

[18]  W. Fischer,et al.  The metric of multidimensional psychological spaces as a function of the differential attention to subjective attributes , 1970 .

[19]  J. Kruskal.  Nonmetric multidimensional scaling: A numerical method , 1964 .

[20]  Patricia Somers,et al.  A Psychophysical Approach to Dimensional Integrality , 1980 .

[21]  I. Barrodale,et al.  An Improved Algorithm for Discrete $l_1 $ Linear Approximation , 1973 .

[22]  Rainer Schrader,et al.  On the Convergence of Stationary Distributions in Simulated Annealing Algorithms , 1988, Inf. Process. Lett..

[23]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[24]  F. Aluffi-Pentini,et al.  Global optimization and stochastic differential equations , 1985 .

[25]  S. Dreyfus,et al.  Thermodynamical Approach to the Traveling Salesman Problem : An Efficient Simulation Algorithm , 2004 .

[26]  E. Rothkopf.  A measure of stimulus similarity and errors in some paired-associate learning tasks. , 1957, Journal of experimental psychology.

[27]  Pieter M. Kroonenberg.  Review of I. Borg & J. Lingoes, "Multidimensional similarity structure analysis" , 1990 .

[28]  P. Groenen,et al.  Global Optimization in Least-Squares Multidimensional Scaling by Distance Smoothing , 1999 .

[29]  F. Rendl,et al.  A thermodynamically motivated simulation procedure for combinatorial optimization problems , 1984 .

[30]  R. Shepard.  The analysis of proximities: Multidimensional scaling with an unknown distance function. I. , 1962 .

[31]  Michael J. Brusco.  A Simulated Annealing Heuristic for Unidimensional and Multidimensional (City-Block) Scaling of Symmetric Proximity Matrices , 2001, J. Classif..

[32]  R. Shepard.  Attention and the metric structure of the stimulus space. , 1964 .

[33]  P. Groenen,et al.  Modern multidimensional scaling , 1996 .

[34]  Willem J. Heiser,et al.  City-Block Scaling: Smoothing Strategies for Avoiding Local Minima , 1998 .

[35]  Rudolf Mathar,et al.  Multidimensional Scaling with ℓp-Distances, a Unifying Approach , 1994 .

[36]  Willem J. Heiser,et al.  A Latent Class Multidimensional Scaling Model for Two-Way One-Mode Continuous Rating Dissimilarity Data , 2009 .

[37]  J. Leeuw.  Applications of Convex Analysis to Multidimensional Scaling , 2000 .

[38]  W. Torgerson,et al.  Multidimensional scaling of similarity , 1965, Psychometrika.

[39]  José A. Díaz-García,et al.  A global simulated annealing heuristic for the three-parameter lognormal maximum likelihood estimation , 2008, Computational Statistics & Data Analysis.

[40]  Charles L. Lawson,et al.  Solving least squares problems , 1976, Classics in applied mathematics.

[41]  Paul E. Green,et al.  Multidimensional Scaling: Concepts and Applications , 1989 .

[42]  R. Shepard.  The analysis of proximities: Multidimensional scaling with an unknown distance function. II , 1962 .

[43]  Karl F. Wender,et al.  A test of independence of dimensions in multidimensional scaling , 1971 .