The Centroid Decomposition: Relationships between Discrete Variational Decompositions and SVDs

The centroid decomposition, an approximation for the singular value decomposition (SVD), has a long history among the statistics/psychometrics community for factor analysis research. We revisit the centroid method in its original context of factor analysis and then adapt it to other than a covariance matrix. The centroid method can be cast as an ${\cal O}(n)$-step ascent method on a hypercube. It is shown empirically that the centroid decomposition provides a measurement of second order statistical information of the original data in the direction of the corresponding left centroid vectors. One major purpose of this work is to show fundamental relationships between the singular value, centroid, and semidiscrete decompositions. This unifies an entire class of truncated SVD approximations. Applications include semantic indexing in information retrieval.