Minimum error classification training of HMMs

This paper describes an implementation of minimum error training for continuous gaussian mixture density HMMs. Instead of maximising the conditional probability of producing a set of training data, as in the conventional HMM maximum likelihood approach, we train to minimise the number of recognition errors. The most important aspect of this work is the use of a first order differentiable “loss” function, the minimisation of which is directly related to the minimisation of the recognition error rate. The performance of the resulting minimum error HMMs was compared against that of conventional maximum likelihood HMMs in a continuous speech recognition task using the ATR 5, 240 Japanese word data base. The results were impressive. For example, for 10 mixture 5 state Baum-Welch trained HMMs, after minimum error training word error rates reduced from 20.6% to 3.0% on the closed training set and 23.2% to 13.2% on the open test set. Furthermore, 3 mixture minimum error HMMs performed better than 10 mixture maximum likelihood HMMs. In fact in every performance measure made the minimum error HMMs proved to be superior.