Latent Variable Models for Dimensionality Reduction

Principal coordinate analysis (PCO), a dual of principal component analysis (PCA), is a classical method for exploratory data analysis. In this paper we provide a probabilistic interpretation of PCO. We show that this interpretation yields a maximum likelihood procedure for estimating the PCO parameters and we also present an iterative expectation-maximization algorithm for obtaining maximum likelihood estimates. Finally, we show that our framework yields a probabilistic formulation of kernel PCA.

[1]  N. L. Johnson,et al.  Multivariate Analysis , 1958, Nature.

[2]  J. Gower.  Some distance properties of latent root and vector methods used in multivariate analysis , 1966 .

[3]  J. Ramsay.  Some Statistical Approaches to Multidimensional Scaling Data , 1982 .

[4]  J. Schmee.  An Introduction to Multivariate Statistical Analysis , 1986 .

[5]  D. Bartholomew.  Latent Variable Models And Factor Analysis , 1987 .

[6]  Patrick J. F. Groenen,et al.  The majorization approach to multidimensional scaling : some problems and extensions , 1993 .

[7]  C. Bishop,et al.  Analysis of multiphase flows using dual-energy gamma densitometry and neural networks , 1993 .

[8]  P. Groenen,et al.  The majorization approach to multidimensional scaling for Minkowski distances , 1995 .

[9]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[10]  P. Groenen,et al.  Modern multidimensional scaling , 1996 .

[11]  Bernhard Schölkopf,et al.  Nonlinear Component Analysis as a Kernel Eigenvalue Problem , 1998, Neural Computation.

[12]  A. Rukhin.  Matrix Variate Distributions , 1999, The Multivariate Normal Distribution.

[13]  Michael E. Tipping,et al.  Probabilistic Principal Component Analysis , 1999 .

[14]  A. Raftery,et al.  Bayesian Multidimensional Scaling and Choice of Dimension , 2001 .

[15]  Stan Lipovetsky,et al.  Latent Variable Models and Factor Analysis , 2001, Technometrics.

[16]  I. Jolliffe.  Principal Component Analysis , 2002 .

[17]  Christopher K. I. Williams.  On a Connection between Kernel PCA and Metric Multidimensional Scaling , 2004, Machine Learning.

[18]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[19]  Ian T. Jolliffe,et al.  Principal Component Analysis , 2002, International Encyclopedia of Statistical Science.

[20]  J. Magnus,et al.  Matrix Differential Calculus with Applications in Statistics and Econometrics , 2019, Wiley Series in Probability and Statistics.