Product of Gaussians as a distributed representation for speech recognition

Distributed representations allow the effective number of Gaussian components in a mixture model, or state of an HMM, to be increased without dramatically increasing the number of model parameters. Various forms of distributed representation have previously been investigated. In this work it shown that the product of experts (PoE) framework may be viewed as a distributed representation when the individual experts are mixtures of Gaussians. However, in contrast to the standard PoE model, the individual experts are not required to be valid distributions, thus allowing additional flexibility in the component priors and variances. The performance of PoE models when used as a distributed representation on a large vocabulary speech recognition task, SwitchBoard, is evaluated.