Factored sparse inverse covariance matrices

Most HMM-based speech recognition systems use Gaussian mixtures as observation probability density functions. An important goal in all such systems is to improve parsimony. One method is to adjust the type of covariance matrices used. In this work, factored sparse inverse covariance matrices are introduced. Based on U'DU factorization, the inverse covariance matrix can be represented using linear regressive coefficients which 1) correspond to sparse patterns in the inverse covariance matrix (and therefore represent conditional independence properties of the Gaussian), and 2), result in a method of partial tying of the covariance matrices without requiring non-linear EM update equations. Results show that the performance of full-covariance Gaussians can be matched by factored sparse inverse covariance Gaussians having significantly fewer parameters.

[1]  Michael I. Jordan.  Graphical Models , 1998 .

[2]  Jeff A. Bilmes,et al.  Buried Markov models for speech recognition , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[3]  R. E. Roger.  Sparse inverse covariance matrices and efficient maximum likelihood classification of hyperspectral data , 1996 .

[4]  Daphne Koller,et al.  Toward Optimal Feature Selection , 1996, ICML.

[5]  Mark J. F. Gales,et al.  Semi-tied covariance matrices for hidden Markov models , 1999, IEEE Trans. Speech Audio Process..

[6]  A. B. Poritz,et al.  Linear predictive hidden Markov models and the speech signal , 1982, ICASSP.

[7]  Hsiao-Chuan Wang,et al.  Joint estimation of feature transformation parameters and Gaussian mixture model for speaker identification , 1999, Speech Commun..

[8]  Ramesh A. Gopinath,et al.  Model selection in acoustic modeling , 1999, EUROSPEECH.

[9]  Bo Thiesson,et al.  Learning Mixtures of Bayesian Networks , 1997, UAI 1997.

[10]  Hong C. Leung,et al.  PhoneBook: a phonetically-rich isolated-word telephone-speech database , 1995, 1995 International Conference on Acoustics, Speech, and Signal Processing.

[11]  Jeff A. Bilmes,et al.  Natural statistical models for automatic speech recognition , 1999 .

[12]  Steve Young,et al.  A review of large-vocabulary continuous-speech , 1996, IEEE Signal Process. Mag..

[13]  Hiroshi Kanazawa,et al.  A flexible method of creating HMM using block-diagonalization of covariance matrices , 1998, ICSLP.

[14]  Andrej Ljolje.  The importance of cepstral parameter correlations in speech recognition , 1994, Comput. Speech Lang..

[15]  Steve Young,et al.  A review of large-vocabulary continuous-speech recognition , 1996 .