Discriminative estimation of subspace precision and mean (SPAM) models

The SPAM model was recently proposed as a very general method for modeling Gaussians with constrained means and covariances. It has been shown to yield significant error rat e improvements over other methods of constraining covarianc es such as diagonal, semi-tied covariances, and extended maxi mum likelihood linear transformations. In this paper we address the problem of discriminative estimation of SPAM mode l parameters, in an attempt to further improve its performanc e. We present discriminative estimation under two criteria: m aximum mutual information (MMI) and an “error-weighted” train ing. We show that both these methods individually result in over 20% relative reduction in word error rate on a digit task over maximum likelihood (ML) estimated SPAM model parameters. We also show that a gain of as much as 28% relative can be achieved by combining these two discriminative estimati on techniques. The techniques developed in this paper also app ly directly to an extension of SPAM called subspace constraine d exponential models.