Multilingual speech recognition with language identification

This paper presents a new approach to multilingual speech recognition. The proposed algorithm combines both language identification (LID) and speech recognition into a single process. It is shown to be effective for multilingual grammarbased speech recognition where the language information is not available prior to recognition. The idea is to make use of acoustic-phonetic and lexical information in each language to reduce possible mismatch caused by potential difference in acoustic and recording conditions when the training utterances for each language were collected. By doing so, it is shown that, with the help of LID information, the word error rate of a mixed Mandarin and English speech recognition system is greatly reduced. The same formulation can also be used to enhance language identification accuracy.