Chinese-English bilingual phone modeling for cross-language speech recognition

In this paper, three different approaches to Chinese-English bilingual phone modeling are investigated and compared. The first approach is to simply combine Chinese and English phone inventories together without phone sharing across the languages. The second one is to map language-dependent phones to the inventory of the International Phonetic Association (IPA) based on phonetic knowledge to construct the bilingual phone inventory. The third one is to merge the language-dependent phone models by an hierarchical phone clustering algorithm to get a compact bilingual inventory. In the third approach, two distance measures are used to perform the bottom-up clustering. One is the Bhattacharyya distance. The other is the acoustic likelihood distance. Experimental results show that the phone clustering approach outperforms the IPA-based phone mapping approach, and it can also achieve comparable performance to the simple combination of language-dependent phone inventories with fewer model parameters, especially when using acoustic likelihood distance measurement.