Selecting Artificially-Generated Sentences for Fine-Tuning Neural Machine Translation

Neural Machine Translation (NMT) models tend to achieve best performance when larger sets of parallel sentences are provided for training. For this reason, augmenting the training set with artificially-generated sentence pairs can boost performance. Nonetheless, the performance can also be improved with a small number of sentences if they are in the same domain as the test set. Accordingly, we want to explore the use of artificially-generated sentences along with data-selection algorithms to improve German-to-English NMT models trained solely with authentic data. In this work, we show how artificially-generated sentences can be more beneficial than authentic pairs, and demonstrate their advantages when used in combination with data-selection algorithms.

[1]  Rico Sennrich,et al.  Improving Neural Machine Translation Models with Monolingual Data , 2015, ACL.

[2]  Philipp Koehn,et al.  Findings of the 2015 Workshop on Statistical Machine Translation , 2015, WMT@EMNLP.

[3]  Maja Popovic,et al.  chrF: character n-gram F-score for automatic MT evaluation , 2015, WMT@EMNLP.

[4]  Germán Sanchis-Trilles,et al.  Does more data always yield better translations? , 2012, EACL.

[5]  Francisco Casacuberta,et al.  Adapting Neural Machine Translation with Parallel Synthetic Data , 2017, WMT.

[6]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[7]  Karin M. Verspoor,et al.  Findings of the WMT 2017 Biomedical Translation Shared Task , 2017, WMT.

[8]  Zuzanna Parcheta,et al.  Data selection for NMT using Infrequent n-gram Recovery , 2018 .

[9]  Alon Lavie,et al.  Better Hypothesis Testing for Statistical Machine Translation: Controlling for Optimizer Instability , 2011, ACL.

[10]  Matthew G. Snover,et al.  A Study of Translation Edit Rate with Targeted Human Annotation , 2006, AMTA.

[11]  Andy Way,et al.  Investigating Backtranslation in Neural Machine Translation , 2018, EAMT.

[12]  Christopher D. Manning,et al.  Stanford Neural Machine Translation Systems for Spoken Language Domains , 2015, IWSLT.

[13]  Christof Monz,et al.  Dynamic Data Selection for Neural Machine Translation , 2017, EMNLP.

[14]  Deniz Yuret,et al.  Instance Selection for Machine Translation using Feature Decay Algorithms , 2011, WMT@EMNLP.

[15]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[16]  Andy Way,et al.  Feature Decay Algorithms for Neural Machine Translation , 2018, EAMT.

[17]  Rico Sennrich,et al.  Neural Machine Translation of Rare Words with Subword Units , 2015, ACL.

[18]  Ralph Weischedel,et al.  A STUDY OF TRANSLATION ERROR RATE WITH TARGETED HUMAN ANNOTATION , 2005 .

[19]  Philipp Koehn,et al.  Statistical Significance Tests for Machine Translation Evaluation , 2004, EMNLP.

[20]  Alon Lavie,et al.  METEOR: An Automatic Metric for MT Evaluation with Improved Correlation with Human Judgments , 2005, IEEvaluation@ACL.

[21]  Ergun Biçici,et al.  Feature Decay Algorithms for Fast Deployment of Accurate Statistical Machine Translation Systems , 2013, WMT@ACL.

[22]  Alexander M. Rush,et al.  OpenNMT: Open-Source Toolkit for Neural Machine Translation , 2017, ACL.

[23]  Jiajun Zhang,et al.  One Sentence One Model for Neural Machine Translation , 2018, LREC.

[24]  Andy Way,et al.  Transductive Data-Selection Algorithms for Fine-Tuning Neural Machine Translation , 2019, PSLT@MTSummit.

[25]  Andy Way,et al.  Combining PBSMT and NMT Back-translated Data for Efficient NMT , 2019, RANLP.

[26]  Andy Way,et al.  Data Selection with Feature Decay Algorithms Using an Approximated Target Side , 2018, IWSLT.

[27]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[28]  Gholamreza Haffari,et al.  Iterative Back-Translation for Neural Machine Translation , 2018, NMT@ACL.

[29]  Andy Way,et al.  Adaptation of Machine Translation Models with Back-translated Data using Transductive Data Selection Methods , 2019, CICLing.

[30]  Yoshua Bengio,et al.  On Using Monolingual Corpora in Neural Machine Translation , 2015, ArXiv.

[31]  Markus Freitag,et al.  Fast Domain Adaptation for Neural Machine Translation , 2016, ArXiv.

[32]  Andy Way,et al.  The ADAPT System Description for the IWSLT 2018 Basque to English Translation Task , 2018, IWSLT.