A statistical discrimination measure for hidden Markov models based on divergence

This paper proposes and evaluates a new statistical discrimination measure for hidden Markov models (HMMs) extending the notion of divergence [1], a measure of average discrimination information originally defined for two probability density functions. The Average Divergence Distance (ADD) is proposed as a statistical discrimination measure between two HMMs, considering the transient behavior of these models. We show the analytical formulation of this discrimination measure, and demonstrate that this quantity is well defined for a left-to-right HMM topology with final non-emitting state, a standard model for basic acoustic units in Automatic Speech Recognition (ASR). Using experiments based on this discrimination measure, it is shown that ADD is a coherent way to evaluate the discrimination dissimilarity between acoustic models.

[1]  Peder A. Olsen,et al.  Theory and practice of acoustic confusability , 2002, Comput. Speech Lang..

[2]  Richard M. Stern,et al.  Structured redefinition of sound units by merging and splitting for improved speech recognition , 2000, INTERSPEECH.

[3]  Joachim Köhler,et al.  Multi-lingual phoneme recognition exploiting acoustic-phonetic similarities of sounds , 1996, Proceeding of Fourth International Conference on Spoken Language Processing. ICSLP '96.

[4]  S. Kullback,et al.  Information Theory and Statistics , 1959 .

[5]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[6]  Alexander H. Waibel,et al.  Selection criteria for hypothesis driven lexical adaptation , 1999, 1999 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings. ICASSP99 (Cat. No.99CH36258).

[7]  Minh N. Do,et al.  Rotation invariant texture characterization and retrieval using steerable wavelet-domain hidden Markov models , 2002, IEEE Trans. Multim..

[8]  M. Do.  Fast approximation of Kullback-Leibler distance for dependence trees and hidden Markov models , 2003, IEEE Signal Processing Letters.

[9]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[10]  Frederick Jelinek,et al.  Statistical methods for speech recognition , 1997 .

[11]  Lin-Shan Lee,et al.  Pronunciation variation analysis based on acoustic and phonemic distance measures with application examples on Mandarin Chinese , 2003, 2003 IEEE Workshop on Automatic Speech Recognition and Understanding (IEEE Cat. No.03EX721).