论文信息 - Odia transliteration engine using moses

Odia transliteration engine using moses

Transliteration is an important Natural Language Processing task. Transliteration is an automatic method for converting words in one language into phonetically equivalent ones in another language. In this paper we use the popular phrase-based SMT techniques for the task of machine transliteration, for Odia-English and Odia-Hindi language pair. We have created two models for syllable based splits (Odia-English, Odia-Hindi) on 50,900 parallel entries and two models for character based splits (Odia-English, Odia-Hindi) on 1,10,000 parallel entries. SRILM is used to build statistical language models. GIZA++ is used to perform word alignments over parallel corpora. We have achieved an accuracy of 89% for Odia-English and 86% for Odia-Hindi on Syllable based split and 71% for Odia-English and 85% for Odia-Hindi on character based split.

Rakesh Chandra Balabantaray | Deepak Sahoo

[1] Kevin Knight,et al. Machine Transliteration , 1997, CL.

[2] Yaser Al-Onaizan,et al. Machine Transliteration of Names in Arabic Texts , 2002, SEMITIC@ACL.

[3] Anil Kumar Singh,et al. A More Discerning and Adaptable Multilingual Transliteration Mechanism for Indian Languages , 2008, IJCNLP.

[4] Hitoshi Isahara,et al. A machine transliteration model based on correspondence between graphemes and phonemes , 2006, TALIP.

[5] Grzegorz Kondrak,et al. Substring-Based Transliteration , 2007, ACL.

[6] Nick Cercone,et al. Computational Linguistics , 1986, Communications in Computer and Information Science.

[7] Peter N. Yianilos,et al. Learning String-Edit Distance , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[8] S. K. Lenka,et al. Name Entity Recognizer for Odia using Conditional Random Fields , 2013 .

[9] Hermann Ney,et al. A Systematic Comparison of Various Statistical Alignment Models , 2003, CL.

[10] Philipp Koehn,et al. Moses: Open Source Toolkit for Statistical Machine Translation , 2007, ACL.

[11] Eiichiro Sumita,et al. Multiple Translation-Engine-based Hypotheses and Edit-Distance-based Rescoring for a Greedy Decoder for Statistical Machine Translation(Natural-Language Processing) , 2005 .

[12] Muhammad Ghulam Abbas Malik,et al. Punjabi Machine Transliteration , 2006, ACL.

[13] Karthik Gali,et al. Modeling Machine Transliteration as a Phrase Based Statistical Machine Translation Problem , 2009, NEWS@IJCNLP.