Improved noise robustness of word HMMs based on weighted variance expansion for noisy speech recognition

In real-life environments such as factories, noise spectra and signal-to-noise ratio can vary dramatically due to the occurrence of nonstationary heavy noise and changes in the surrounding environment. Since the performance of current speech recognition technology noticeably deteriorates when the type of noise and signal-to-noise ratio at recognition time differ from those observed in training or during adaptation, HMMs that are robust to variation in noise are needed. In order to improve the robustness of existing word HMMs to noise, in this paper we propose a method to prevent the output distributions at states that are susceptible to noise from varying significantly on the basis of differences within the observation vectors by performing weighted variance expansions for the states and distributions according to their respective power values. We evaluate the effectiveness of the proposed method on a speaker-independent word recognition task with a vocabulary of 50 words and two types of factory noise, using both clean speech and noisy speech HMMs in recognition settings with different noise conditions (i.e., type of additive noise and signal-to-noise ratio) to those observed in training. Our experimental results show that for all types of HMM considered, recognition rates when there are variations in the noise conditions improved over a wide range of signal-to-noise ratios, confirming that for small-vocabulary noisy speech recognition the robustness of HMMs to noise has been improved by our method. In particular, we note a significant improvement in recognition performance using weighted variance expansions rather than a constant expansion when the signal-to-noise ratio is lower than that observed in training. © 2005 Wiley Periodicals, Inc. Syst Comp Jpn, 36(13): 57–68, 2005; Published online in Wiley InterScience (). DOI 10.1002sscj.20290