Neural Machine Translation Techniques for Named Entity Transliteration

Transliterating named entities from one language into another can be approached as neural machine translation (NMT) problem, for which we use deep attentional RNN encoder-decoder models. To build a strong transliteration system, we apply well-established techniques from NMT, such as dropout regularization, model ensembling, rescoring with right-to-left models, and back-translation. Our submission to the NEWS 2018 Shared Task on Named Entity Transliteration ranked first in several tracks.

[1]  Lemao Liu,et al.  Target-Bidirectional Neural Models for Machine Transliteration , 2016, NEWS@ACM.

[2]  Wang Yinqua.  Chinese Translation of Foreign Personal Names , 2014 .

[3]  Tomi Kinnunen,et al.  INTERSPEECH 2013 14thAnnual Conference of the International Speech Communication Association , 2013, Interspeech 2015.

[4]  Marcin Junczys-Dowmunt,et al.  Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing , 2016, WMT.

[5]  Geoffrey E. Hinton,et al.  Layer Normalization , 2016, ArXiv.

[6]  Rico Sennrich,et al.  The University of Edinburgh’s Neural MT Systems for WMT17 , 2017, WMT.

[7]  Haizhou Li,et al.  Whitepaper of NEWS 2016 Shared Task on Machine Transliteration , 2016, NEWS@ACM.

[8]  Rico Sennrich,et al.  The AMU-UEDIN Submission to the WMT16 News Translation Task: Attention-based NMT Models as Feature Functions in Phrase-based SMT , 2016, WMT.

[9]  Rico Sennrich,et al.  Deep architectures for Neural Machine Translation , 2017, WMT.

[10]  Jian Su,et al.  A Joint Source-Channel Model for Machine Transliteration , 2004, ACL.

[11]  Kevin Knight,et al.  Machine Transliteration , 1997, CL.

[12]  Ahmed Guessoum,et al.  Arabic Machine Transliteration using an Attention-based Encoder-decoder Model , 2017, ACLING.

[13]  Ted Briscoe,et al.  Grammatical error correction using neural machine translation , 2016, NAACL.

[14]  Yoav Goldberg,et al.  Morphological Inflection Generation with Hard Monotonic Attention , 2016, ACL.

[15]  Bin Ma,et al.  Phonology-augmented statistical transliteration for low-resource languages , 2015, INTERSPEECH.

[16]  Alexander M. Rush,et al.  Abstractive Sentence Summarization with Attentive Recurrent Neural Networks , 2016, NAACL.

[17]  Rico Sennrich,et al.  Edinburgh Neural Machine Translation Systems for WMT 16 , 2016, WMT.

[18]  Rico Sennrich,et al.  Neural Machine Translation of Rare Words with Subword Units , 2015, ACL.

[19]  Graham Neubig,et al.  Stronger Baselines for Trustable Results in Neural Machine Translation , 2017, NMT@ACL.

[20]  Nhut M. Pham,et al.  Comparative analysis of transliteration techniques based on statistical machine translation and joint-sequence model , 2010, SoICT.

[21]  Lior Wolf,et al.  Using the Output Embedding to Improve Language Models , 2016, EACL.

[22]  Zoubin Ghahramani,et al.  A Theoretically Grounded Application of Dropout in Recurrent Neural Networks , 2015, NIPS.

[23]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[24]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[25]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[26]  Nadir Durrani,et al.  Integrating an Unsupervised Transliteration Model into Statistical Machine Translation , 2014, EACL.

[27]  Rico Sennrich,et al.  Improving Neural Machine Translation Models with Monolingual Data , 2015, ACL.

[28]  Falk Scholer,et al.  English to Persian Transliteration , 2006, SPIRE.

[29]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[30]  Falk Scholer,et al.  Corpus Effects on the Evaluation of Automated Transliteration Systems , 2007, ACL.

[31]  Mirella Lapata,et al.  Paraphrasing Revisited with Neural Machine Translation , 2017, EACL.

[32]  Rico Sennrich,et al.  Nematus: a Toolkit for Neural Machine Translation , 2017, EACL.