Fast Semi-Supervised Discriminative Component Analysis

We introduce a method that learns a class-discriminative subspace or discriminative components of data. Such a sub- space is useful for visualization, dimensionality reduction, feature extraction, and for learning a regularized distance metric. We learn the subspace by optimizing a probabilistic semiparametric model, a mixture of Gaussians, of classes in the subspace. The semiparametric modeling leads to fast computation (O(N) for N samples) in each iteration of optimization, in contrast to recent nonparametric methods that take O(N2) time, but with equal accuracy. Moreover, we learn the subspace in a semi-supervised manner from three kinds of data: labeled and unlabeled samples, and unlabeled samples with pairwise constraints, with a unified objective.