Preserving Local Structure in Gaussian Process Latent Variable Models

The Gaussian Process Latent Variable Model (GPLVM) is a non-linear variant of probabilistic Principal Components Analysis (PCA). The main advantage of the GPLVM over probabilistic PCA is that it can model non-linear transformations from the latent space to the data space. An important disadvantage of the GPLVM is its focus on preserving global data structure in the latent space, whereas preserving local data structure is generally considered to be more important in dimensionality reduction. In this paper, we present an extension of the GPLVM that encourages the preservation of local structure in the latent space. The extension entails the introduction of a prior distribution over the parameters of the GPLVM that measures the divergence between the pairwise distances in the data space and the latent space. We show that the proposed extension leads to strong results.