Reliable utterance segment recognition by integrating a grammar with statistical language constraints

This paper proposes a novel approach to the recognition of complete utterances and partial segments of utterances. This approach ensures a high level of confidence in the results. The proposed method is based on the cooperative use of a conventional n-gram constraint and additional grammatical constraints which take deviations from the grammar into account with a multi-pass search strategy. The partial utterance segments are obtained with high confidence as the segments that satisfy both n-gram and grammatical constraints. For improved efficiency, the context-free grammar expressing the grammatical constraints is approximated by a finite-state automaton. We consider all kinds of deviations from the grammar such as insertions, deletions and substitutions when applying the grammatical constraints. As a result, we can achieve a more robust application of grammatical constraints compared to a conventional word-skipping robust parser that can only handle one type of deviation, that is, insertions. Our experiments confirm that the proposed method can recognize partial segments of utterances more reliably than conventional continuous speech recognition methods using only n-grams. In addition, our results indicate that allowing more deviations from the grammatical constraints leads to better performance than the conventional word-skipping robust parser approach.