Iterative Refinement for Machine Translation

Existing machine translation decoding algorithms generate translations in a strictly monotonic fashion and never revisit previous decisions. As a result, earlier mistakes cannot be corrected at a later stage. In this paper, we present a translation scheme that starts from an initial guess and then makes iterative improvements that may revisit previous decisions. We parameterize our model as a convolutional neural network that predicts discrete substitutions to an existing translation based on an attention mechanism over both the source sentence as well as the current translation output. By making less than one modification per sentence, we improve the output of a phrase-based translation system by up to 0.4 BLEU on WMT15 German-English translation.

[1]  Christian Federmann,et al.  Proceedings of the Tenth Workshop on Statistical Machine Translation, WMT@EMNLP 2015, 17-18 September 2015, Lisbon, Portugal , 2015, WMT@EMNLP.

[2]  Ronan Collobert,et al.  Word Embeddings through Hellinger PCA , 2013, EACL.

[3]  Alex Kulesza,et al.  Confidence Estimation for Machine Translation , 2004, COLING.

[4]  Hermann Ney,et al.  Word-Level Confidence Estimation for Machine Translation , 2007, CL.

[5]  Daniel Marcu,et al.  What’s in a translation rule? , 2004, NAACL.

[6]  Clément Farabet,et al.  Torch7: A Matlab-like Environment for Machine Learning , 2011, NIPS 2011.

[7]  Liang Huang,et al.  Forest-based algorithms in natural language processing , 2008 .

[8]  Daniel Marcu,et al.  Statistical Phrase-Based Translation , 2003, NAACL.

[9]  Karin M. Verspoor,et al.  Findings of the 2016 Conference on Machine Translation , 2016, WMT.

[10]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[11]  Michel Simard,et al.  Statistical Phrase-Based Post-Editing , 2007, NAACL.

[12]  Philipp Koehn,et al.  Moses: Open Source Toolkit for Statistical Machine Translation , 2007, ACL.

[13]  Christopher D. Manning,et al.  Effective Approaches to Attention-based Neural Machine Translation , 2015, EMNLP.

[14]  Jason Weston,et al.  A Neural Attention Model for Sentence Summarization , 2015 .

[15]  Philipp Koehn,et al.  Word Lattices for Multi-Source Translation , 2009, EACL.

[16]  Ronan Collobert,et al.  Neural Network-based Word Alignment through Score Aggregation , 2016, WMT.

[17]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.