Exploring Word Sense Disambiguation Abilities of Neural Machine Translation Systems (Non-archival Extended Abstract)

Neural machine translation systems have been shown to achieve state-of-the-art translation performance for many language pairs. In order to produce a correct translation, MT systems must learn how to disambiguate words with multiple senses and pick the correct translation. We explore the extent to which the word embeddings for ambiguous words are able to disambiguate senses at deeper layers of the NMT encoder, which are thought to represent words with surrounding context. Consistent with previous research, we find that the NMT system fails to translate many ambiguous words correctly. We provide an evaluation framework to use for proposed improvements to word sense disambiguation abilities of NMT systems.

[1]  Yang Liu,et al.  Visualizing and Understanding Neural Machine Translation , 2017, ACL.

[2]  Hwee Tou Ng,et al.  Word Sense Disambiguation Improves Statistical Machine Translation , 2007, ACL.

[3]  Laura Mascarell,et al.  Improving Word Sense Disambiguation in Neural Machine Translation with Sense Embeddings , 2017, WMT.

[4]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[5]  Philipp Koehn,et al.  Europarl: A Parallel Corpus for Statistical Machine Translation , 2005, MTSUMMIT.

[6]  Yonatan Belinkov,et al.  What do Neural Machine Translation Models Learn about Morphology? , 2017, ACL.

[7]  Philipp Koehn,et al.  Six Challenges for Neural Machine Translation , 2017, NMT@ACL.

[8]  Philipp Koehn,et al.  Moses: Open Source Toolkit for Statistical Machine Translation , 2007, ACL.

[9]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[10]  Daphne Koller,et al.  Word-Sense Disambiguation for Machine Translation , 2005, HLT.

[11]  Marine Carpuat,et al.  Improving Statistical Machine Translation Using Word Sense Disambiguation , 2007, EMNLP.

[12]  Fei-Fei Li,et al.  Visualizing and Understanding Recurrent Networks , 2015, ArXiv.

[13]  Frederick Liu,et al.  Handling Homographs in Neural Machine Translation , 2017, NAACL.

[14]  Marine Carpuat.  A Semantic Evaluation of Machine Translation Lexical Choice , 2013, SSST@NAACL-HLT.

[15]  Alexander M. Rush,et al.  OpenNMT: Open-Source Toolkit for Neural Machine Translation , 2017, ACL.

[16]  Emmanuel Dupoux,et al.  Assessing the Ability of LSTMs to Learn Syntax-Sensitive Dependencies , 2016, TACL.

[17]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.