论文信息 - A stream-weight optimization method for multi-stream HMMs based on likelihood value normalization

A stream-weight optimization method for multi-stream HMMs based on likelihood value normalization

In the field of audio-visual speech recognition, multi-stream HMM are widely used, thus how to automatically and properly determine stream weight factors using a small data set becomes an important research issue. This paper proposes a new stream-weight optimization method based on an output likelihood normalization criterion. In this method, the stream weights are adjusted to equalize the mean values of log likelihood for all HMM based on likelihood-ratio maximization which achieved significant improvement by using a large optimization data set. The new method is evaluated using Japanese connected digit speech recorded in real-world environments. Using 10 seconds speech data for stream-weight optimization, a 10% absolute accuracy improvement is achieved compared to the result before optimization. By additionally applying the MLLR (maximum likelihood linear regression) adaptation, a 23% improvement is obtained over the audio-only scheme.

Sadaoki Furui | Satoshi Tamura | Koji Iwano

[1] John B. Shoven,et al. I , Edinburgh Medical and Surgical Journal.

[2] Gerasimos Potamianos,et al. Speaker independent audio-visual database for bimodal ASR , 1997, AVSP.

[3] Satoshi Nakamura,et al. Stream weight optimization of speech and lip image sequence for audio-visual speech recognition , 2000, INTERSPEECH.

[4] Keiichi Tokuda,et al. Audio-visual speech recognition using MCE-based hmms and model-dependent stream weights , 2000, INTERSPEECH.

[5] Sadaoki Furui,et al. Audio-visual speech recognition using lip movement extracted from side-face images , 2003, AVSP.

[6] Sadaoki Furui,et al. A Robust Multimodal Speech Recognition Method using Optical Flow Analysis , 2005 .

[7] Sadaoki Furui,et al. A stream-weight optimization method for audio-visual speech recognition using multi-stream HMMs , 2004, 2004 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[8] Philip C. Woodland,et al. Maximum likelihood linear regression for speaker adaptation of continuous density hidden Markov models , 1995, Comput. Speech Lang..