Extending metric multidimensional scaling with Bregman divergences

Sum of weighted square distance errors has been a popular way of defining stress function for metric multidimensional scaling (MMDS) like the Sammon mapping. In this paper we generalise this popular MMDS with Bregman divergences, as an example we show that the Sammon mapping can be thought of as a truncated Bregman MMDS (BMMDS) and we show that the full BMMDS improves upon the Sammon mapping on some standard data sets and investigate the reasons underlying this improvement. We then extend a well known family of MMDS, that deploy a strategy of focusing on small distances, with BMMDS and investigate limitations of the strategy empirically. Then an opposite strategy is introduced to create another family of BMMDS that gives increasing mapping quality. A data preprocessing method and a distance matrix preprocessing are introduced.

[1]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[2]  Michel Verleysen,et al.  Nonlinear Dimensionality Reduction , 2021, Computer Vision.

[3]  Colin Fyfe,et al.  Curvilinear component analysis and Bregman divergences , 2010, ESANN.

[4]  Li Yang.  Sammon's nonlinear mapping using geodesic distances , 2004, ICPR 2004.

[5]  Jeanny Hérault,et al.  Curvilinear Component Analysis for High-Dimensional Data Representation: I. Theoretical Aspects and Practical Use in the Presence of Noise , 1999, IWANN.

[6]  A. Buja,et al.  Local Multidimensional Scaling for Nonlinear Dimension Reduction, Graph Drawing, and Proximity Analysis , 2009 .

[7]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[8]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[9]  H. Hotelling.  Analysis of a complex of statistical variables into principal components. , 1933 .

[10]  Andreas Buja,et al.  Visualization Methodology for Multidimensional Scaling , 2002, J. Classif..

[11]  John W. Sammon,et al.  A Nonlinear Mapping for Data Structure Analysis , 1969, IEEE Transactions on Computers.

[12]  Richard C. T. Lee,et al.  A Triangulation Method for the Sequential Mapping of Points from N-Space to Two-Space , 1977, IEEE Transactions on Computers.

[13]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.

[14]  Anil K. Jain,et al.  Artificial neural networks for feature extraction and multivariate data projection , 1995, IEEE Trans. Neural Networks.

[15]  Dennis Child,et al.  The essentials of factor analysis , 1970 .

[16]  Jeanny Hérault,et al.  Curvilinear component analysis: a self-organizing neural network for nonlinear mapping of data sets , 1997, IEEE Trans. Neural Networks.