Gated Module Neural Network for Multilingual Speech Recognition

For most multilingual large vocabulary continuous speech recognition (LVCSR) systems, when multiple languages are allowed at the same time, their performance will degrade significantly due to the strong inter-language competition in the decoding phase. To increase the inter-language discrimination capacity, this paper presents a gated module neural network (GMN) approach that adapts a language identification (LID) component to directly assist the final multilingual LVCSR goal. Thanks to an international collaboration 3 large-scale speech corpora (Mandarin, English and Slovak, denoted as Zh, En and Sk) were shared for studying this problem. Hence the proposed approach was evaluated on both bilingual (Zh/En and Sk/En) and trilingual (Zh/En/Sk) LVCSR tasks. The experimental results show that the proposed GMN is promising and the performance of multilingual LVCSRs are now more comparable with the monolingual ones.