HW-TSC’s Participation at WMT 2020 Automatic Post Editing Shared Task

The paper presents the submission by HWTSC in the WMT 2020 Automatic Post Editing Shared Task. We participate in the English→German and English→Chinese language pairs. Our system is built based on the Transformer pre-trained on WMT 2019 and WMT 2020 News Translation corpora, and fine-tuned on the APE corpus. Bottleneck Adapter Layers are integrated into the model to prevent over-fitting. We further collect external translations as the augmented MT candidates to improve the performance. The experiment demonstrates that pre-trained NMT models are effective when fine-tuning with the APE corpus of a limited size, and the performance can be further improved with external MT augmentation. Our system achieves competitive results on both directions in the final evaluation.

[1]  Hao Yang,et al.  Efficient Transfer Learning for Quality Estimation with Bottleneck Adapter Layer , 2020, EAMT.

[2]  Josef van Genabith,et al.  A Neural Network based Approach to Automatic Post-Editing , 2016, ACL.

[3]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[4]  Marco Turchi,et al.  ESCAPE: a Large-scale Synthetic Corpus for Automatic Post-Editing , 2018, LREC.

[5]  Salim Roukos,et al.  Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[6]  Marcin Junczys-Dowmunt,et al.  The AMU-UEdin Submission to the WMT 2017 Shared Task on Automatic Post-Editing , 2017, WMT.

[7]  André F. T. Martins,et al.  Unbabel’s Submission to the WMT2019 APE Shared Task: BERT-Based Encoder-Decoder for Automatic Post-Editing , 2019, WMT.

[8]  Matthew G. Snover,et al.  A Study of Translation Edit Rate with Targeted Human Annotation , 2006, AMTA.

[9]  Marcin Junczys-Dowmunt,et al.  Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing , 2016, WMT.

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[12]  Ying Qin,et al.  NMT Enhancement based on Knowledge Graph Mining with Pre-trained Language Model , 2020, 2020 22nd International Conference on Advanced Communication Technology (ICACT).

[13]  Marcin Junczys-Dowmunt,et al.  An Exploration of Neural Sequence-to-Sequence Architectures for Automatic Post-Editing , 2017, IJCNLP.

[14]  Myle Ott,et al.  fairseq: A Fast, Extensible Toolkit for Sequence Modeling , 2019, NAACL.

[15]  Hao Yang,et al.  Domain Specific NMT based on Knowledge Graph Embedding and Attention , 2019, 2019 21st International Conference on Advanced Communication Technology (ICACT).

[16]  Mona Attariyan,et al.  Parameter-Efficient Transfer Learning for NLP , 2019, ICML.

[17]  Markus Freitag,et al.  Findings of the WMT 2020 Shared Task on Automatic Post-Editing , 2020, WMT.

[18]  Matteo Negri,et al.  Findings of the WMT 2019 Shared Task on Automatic Post-Editing , 2019, WMT.

[19]  Yang Liu,et al.  THUMT: An Open-Source Toolkit for Neural Machine Translation , 2017, AMTA.