Finding consensus in speech recognition

This thesis explores new ways of utilizing the information existing in word lattices produced by speech recognition systems to improve the accuracy of the recognition output and obtain a more perspicuous representation of a set of alternative hypotheses. We change the standard problem formulation of searching among a large set of sentence hypotheses to a local search in a small set of word candidates. Our approach replaces sentence-level posterior probabilities with word-level posteriors as the objective function for speech recognition, corresponding to the word-based error metric commonly used. The core of the method is a clustering procedure that identifies mutually supporting and competing word hypotheses in a lattice, constructing a total order over all word hypotheses. Together with word posterior probabilities computed from recognizer scores, this allows an efficient extraction of the hypothesis that is expected to minimize the word error rate. Our approach thus overcomes the mismatch between the word-based performance metric and the standard MAP scoring paradigm which is sentence-based, that can lead to sub-optimal recognition results. We also show that our method can be used as an efficient lattice compression technique. Its success comes from the ability to discard links with low a posteriori probability and recombine the remaining ones to create a new set of hypotheses. Experiments on the Switchboard corpus and Broadcast News show that this approach results in significant word error rate reductions, both over the standard MAP approach and compared to a previous word error minimization technique based on N-best lists. We also report significant decrease in lattice size when compared with the conventionally used technique. In essence, our method is an estimator of word posterior probabilities, and as such could benefit a number of other tasks like word spotting and confidence annotation.