UCAM Biomedical Translation at WMT19: Transfer Learning Multi-domain Ensembles

The 2019 WMT Biomedical translation task involved translating Medline abstracts. We approached this using transfer learning to obtain a series of strong neural models on distinct domains, and combining them into multi-domain ensembles. We further experiment with an adaptive language-model ensemble weighting scheme. Our submission achieved the best submitted results on both directions of English-Spanish.

[1]  Adrià de Gispert,et al.  Multi-representation ensembles and delayed SGD updates improve syntax-based NMT , 2018, ACL.

[2]  Jia Xu,et al.  Hunter NMT System for WMT18 Biomedical Translation Task: Transfer Learning in Neural Machine Translation , 2018, WMT.

[3]  Kenneth Heafield,et al.  KenLM: Faster and Smaller Language Model Queries , 2011, WMT@EMNLP.

[4]  Samy Bengio,et al.  Tensor2Tensor for Neural Machine Translation , 2018, AMTA.

[5]  Rico Sennrich,et al.  The AMU-UEDIN Submission to the WMT16 News Translation Task: Attention-based NMT Models as Feature Functions in Phrase-based SMT , 2016, WMT.

[6]  Christopher D. Manning,et al.  Stanford Neural Machine Translation Systems for Spoken Language Domains , 2015, IWSLT.

[7]  Deniz Yuret,et al.  Transfer Learning for Low-Resource Neural Machine Translation , 2016, EMNLP.

[8]  Cyril Allauzen,et al.  Bayesian Language Model Interpolation for Mobile Speech Input , 2011, INTERSPEECH.

[9]  Karin M. Verspoor,et al.  Findings of the WMT 2017 Biomedical Translation Shared Task , 2017, WMT.

[10]  Matt Post,et al.  A Call for Clarity in Reporting BLEU Scores , 2018, WMT.

[11]  Mariana L. Neves,et al.  The Scielo Corpus: a Parallel Corpus of Scientific Publications for Biomedicine , 2016, LREC.

[12]  Adrià de Gispert,et al.  CUED@WMT19:EWC&LMs , 2019, WMT.

[13]  Josep Maria Crego,et al.  Domain Control for Neural Machine Translation , 2016, RANLP.

[14]  Pavel Pecina,et al.  Khresmoi Summary Translation Test Data 1.1 , 2014 .

[15]  Rico Sennrich,et al.  Neural Machine Translation of Rare Words with Subword Units , 2015, ACL.

[16]  Markus Freitag,et al.  Fast Domain Adaptation for Neural Machine Translation , 2016, ArXiv.

[17]  Chenhui Chu,et al.  An Empirical Comparison of Domain Adaptation Methods for Neural Machine Translation , 2017, ACL.

[18]  Bill Byrne,et al.  SGNMT – A Flexible NMT Decoding Platform for Quick Prototyping of New Models and Search Strategies , 2017, EMNLP.

[19]  Bill Byrne,et al.  Domain Adaptive Inference for Neural Machine Translation , 2019, ACL.