论文信息 - Findings of the 2020 Conference on Machine Translation (WMT20) - 字舞流文

Findings of the 2020 Conference on Machine Translation (WMT20)

This paper presents the results of the news translation task and the similar language translation task, both organised alongside the Conference on Machine Translation (WMT) 2020. In the news task, participants were asked to build machine translation systems for any of 11 language pairs, to be evaluated on test sets consisting mainly of news stories. The task was also opened up to additional test suites to probe specific aspects of translation. In the similar language translation task, participants built machine translation systems for translating between closely related pairs of languages.

Philipp Koehn | Matt Post | Santanu Pal | Matthias Huck | Masaaki Nagata | Christof Monz | Christian Federmann | Loïc Barrault | Nikola Ljubešić | Ondřej Bojar | Barry Haddow | Marcos Zampieri | Chi-kiu Lo | Roman Grundkiewicz | Makoto Morishita | Magdalena Biesialska | Tom Kocmi | Toshiaki Nakazawa | Yvette Graham | Eric Joanis | Marta R. Costa-jussà | Philipp Koehn | Christof Monz | Matt Post | Ondrej Bojar | Loïc Barrault | B. Haddow | C. Federmann | Toshiaki Nakazawa | Roman Grundkiewicz | Nikola Ljubesic | Yvette Graham | Marcos Zampieri | M. Costa-jussà | Magdalena Biesialska | Tom Kocmi | Makoto Morishita | M. Nagata | Santanu Pal | Chi-kiu Lo | E. Joanis | Matthias Huck | Chi-kiu (羅致翹) Lo

[1] Philipp Koehn,et al. Statistical Power and Translationese in Machine Translation Evaluation , 2020, EMNLP.

[2] Alexandra Birch,et al. The University of Edinburgh's English-Tamil and English-Inuktitut Submissions to the WMT20 News Translation Task , 2020, WMT@EMNLP.

[3] Has Neural Machine Translation Achieved Human Parity ? , 2018 .

[4] Philipp Koehn,et al. Translationese in Machine Translation Evaluation , 2019, EMNLP.

[5] Shiliang Sun,et al. HW-TSC's Participation in the WMT 2020 News Translation Shared Task , 2020, WMT@EMNLP.

[6] Barry Haddow,et al. PMIndia - A Collection of Parallel Corpora of Languages of India , 2020, ArXiv.

[7] Markus Freitag,et al. Results of the WMT20 Metrics Shared Task , 2020, WMT.

[8] Marta R. Costa-jussà,et al. Findings of the 2019 Conference on Machine Translation (WMT19) , 2019, WMT.

[9] Tom Kocmi. CUNI Submission for the Inuktitut Language in WMT News 2020 , 2020, WMT@EMNLP.

[10] Xiaoheng Zhang,et al. Dialect MT: A Case Study between Cantonese and Mandarin , 1998, COLING-ACL.

[11] Marcin Chochowski,et al. Samsung R&D Institute Poland submission to WMT20 News Translation Task , 2020, WMT@EMNLP.

[12] Martin Popel. CUNI English-Czech and English-Polish Systems in WMT20: Robust Document-Level Training , 2020, WMT@EMNLP.

[13] Maite Oronoz,et al. Findings of the WMT 2020 Biomedical Translation Shared Task: Basque, Italian and Russian as New Additional Languages , 2020, WMT.

[14] Hans Uszkoreit,et al. Linguistic Evaluation of German-English Machine Translation Using a Test Suite , 2019, WMT.

[15] Graham Neubig,et al. TICO-19: the Translation Initiative for Covid-19 , 2020, NLP4COVID@EMNLP.

[16] Yang Liu,et al. THUMT: An Open-Source Toolkit for Neural Machine Translation , 2017, AMTA.

[17] Timothy Baldwin,et al. Is Machine Translation Getting Better over Time? , 2014, EACL.

[18] Philipp Koehn,et al. Manual and Automatic Evaluation of Machine Translation between European Languages , 2006, WMT@HLT-NAACL.

[19] Jörg Tiedemann,et al. The OPUS Corpus - Parallel and Free: http://logos.uio.no/opus , 2004, LREC.

[20] Shuming Shi,et al. Tencent Neural Machine Translation Systems for the WMT20 News Translation Task , 2020, WMT.

[21] Yves Scherrer,et al. The University of Helsinki and Aalto University submissions to the WMT 2020 news and low-resource translation tasks , 2020, WMT@EMNLP.

[22] Amit Kumar,et al. Transformer-based Neural Machine Translation System for Hindi – Marathi: WMT20 Shared Task , 2020, WMT.

[23] Marta R. Costa-jussà,et al. A Neural Approach to Language Variety Translation , 2018, VarDial@COLING 2018.

[24] Philipp Koehn,et al. Moses: Open Source Toolkit for Statistical Machine Translation , 2007, ACL.

[25] Sheila Castilho,et al. On Context Span Needed for Machine Translation Evaluation , 2020, LREC.

[26] Anja Belz,et al. The Second Multilingual Surface Realisation Shared Task (SR'19): Overview and Evaluation Results , 2019, MSR@EMNLP-IJCNLP.

[27] Maja Popovic,et al. On reducing translation shifts in translations intended for MT evaluation , 2019, MTSummit.

[28] Matt Post,et al. The Language Demographics of Amazon Mechanical Turk , 2014, TACL.

[29] Philipp Koehn,et al. (Meta-) Evaluation of Machine Translation , 2007, WMT@ACL.

[30] Lucia Specia,et al. Findings of the WMT 2020 Shared Task on Quality Estimation , 2020, WMT.

[31] Wang Ling,et al. BP2EP - Adaptation of Brazilian Portuguese texts to European Portuguese , 2011, EAMT.

[32] Pushpak Bhattacharyya,et al. IndoWordNet , 2010, LREC.

[33] C. V. Jawahar,et al. A Multilingual Parallel Corpora Collection Effort for Indian Languages , 2020, LREC.

[34] Srivatsan Srinivasan,et al. The DeepMind Chinese–English Document Translation System at WMT2020 , 2020, WMT.

[35] Arianna Bisazza,et al. Linguistically Motivated Subwords for English-Tamil Translation: University of Groningen's Submission to WMT-2020 , 2020, WMT@EMNLP.

[36] Philipp Koehn,et al. Findings of the 2013 Workshop on Statistical Machine Translation , 2013, WMT@ACL.

[37] Philipp Koehn,et al. Findings of the 2012 Workshop on Statistical Machine Translation , 2012, WMT@NAACL-HLT.

[38] Philipp Koehn,et al. Findings of the 2011 Workshop on Statistical Machine Translation , 2011, WMT@EMNLP.

[39] Eleftherios Avramidis,et al. Fine-grained linguistic evaluation for state-of-the-art Machine Translation , 2020, WMT.

[40] Benjamin Marie,et al. Combination of Neural Machine Translation Systems at WMT20 , 2020, WMT.

[41] Muhammad Abdul-Mageed,et al. Translating Similar Languages: Role of Mutual Intelligibility in Multilingual Transformers , 2020, WMT.

[42] Noah A. Smith,et al. Evaluating Gender Bias in Machine Translation , 2019, ACL.

[43] Timothy Baldwin,et al. Can machine translation systems be evaluated by the crowd alone , 2015, Natural Language Engineering.

[44] Maria Leonor Pacheco,et al. of the Association for Computational Linguistics: , 2001 .

[45] Jeremy Gwinnup,et al. The AFRL WMT20 News Translation Systems , 2020, WMT@EMNLP.

[46] Salim Roukos,et al. Bleu: a Method for Automatic Evaluation of Machine Translation , 2002, ACL.

[47] Martin Popel,et al. Transforming machine translation: a deep learning system reaches news translation quality comparable to human professionals , 2020, Nature Communications.

[48] Mauro Cettolo,et al. WIT3: Web Inventory of Transcribed and Translated Talks , 2012, EAMT.

[49] Marcis Pinnis,et al. Tilde at WMT 2020: News Task Systems , 2020, WMT@EMNLP.

[50] Guillaume Lample,et al. Cross-lingual Language Model Pretraining , 2019, NeurIPS.

[51] Marta R. Costa-jussa,et al. MT-Adapted Datasheets for Datasets: Template and Repository , 2020, ArXiv.

[52] Jie Zhou,et al. WeChat Neural Machine Translation Systems for WMT20 , 2020, WMT@EMNLP.

[53] Yunpeng Li,et al. IIE’s Neural Machine Translation Systems for WMT20 , 2020, WMT.

[54] Anoop Kunchukuttan,et al. Contact Relatedness can help improve multilingual NMT: Microsoft STCI-MT @ WMT20 , 2020, WMT.

[55] Timothy Baldwin,et al. Further Investigation into Reference Bias in Monolingual Evaluation of Machine Translation , 2017, EMNLP.

[56] Jun Suzuki,et al. JParaCrawl: A Large Scale Web-Based English-Japanese Parallel Corpus , 2020, LREC.

[57] Nikola Ljubesic,et al. {bs,hr,sr}WaC - Web Corpora of Bosnian, Croatian and Serbian , 2014, WaC@EACL.

[58] Alberto Poncelas,et al. Neural Machine Translation between similar South-Slavic languages , 2020, WMT@EMNLP.

[59] Martin Popel,et al. CUNI Transformer Neural MT System for WMT18 , 2018, WMT.

[60] Christian Federmann,et al. Appraise: an Open-Source Toolkit for Manual Evaluation of MT Output , 2012, Prague Bull. Math. Linguistics.

[61] Xiaopu Li,et al. OPPO's Machine Translation Systems for WMT20 , 2020, WMT@EMNLP.

[62] Philipp Koehn,et al. The FLORES Evaluation Datasets for Low-Resource Machine Translation: Nepali–English and Sinhala–English , 2019, EMNLP.

[63] Philipp Koehn,et al. Findings of the 2010 Joint Workshop on Statistical Machine Translation and Metrics for Machine Translation , 2010, WMT@ACL.

[64] Tomaz Erjavec,et al. The slWaC Corpus of the SloveneWeb , 2015, Informatica.

[65] Qingmin Liu,et al. GTCOM Neural Machine Translation Systems for WMT20 , 2020, WMT@EMNLP.

[66] Bharathi Raja Chakravarthi,et al. NUIG-Panlingua-KMI Hindi-Marathi MT Systems for Similar Language Translation Task @ WMT 2020 , 2020, WMT.

[67] Sivaji Bandyopadhyay,et al. Hindi-Marathi Cross Lingual Model , 2020, WMT.

[68] Grigori Sidorov,et al. The IPN-CIC team system submission for the WMT 2020 similar language task , 2020, WMT.

[69] Jörg Tiedemann,et al. News from OPUS — A collection of multilingual parallel corpora with tools and interfaces , 2009 .

[70] Jingbo Zhu,et al. The NiuTrans Machine Translation Systems for WMT20 , 2021, WMT.

[71] Alexander Molchanov. PROMT Systems for WMT 2020 Shared News Translation Task , 2020, WMT@EMNLP.

[72] Yoonjung Choi,et al. An Iterative Knowledge Transfer NMT System for WMT20 News Translation Task , 2020, WMT@EMNLP.

[73] Martin Wattenberg,et al. Google’s Multilingual Neural Machine Translation System: Enabling Zero-Shot Translation , 2016, TACL.

[74] Miquel Espl,et al. Bitextor, a free/open-source software to harvest translation memories from multilingual websites , 2009 .

[75] Leo Wanner,et al. The First Multilingual Surface Realisation Shared Task (SR’18): Overview and Evaluation Results , 2018 .

[76] Dipti Sharma,et al. NMT based Similar Language Translation for Hindi - Marathi , 2020, WMT@EMNLP.

[77] Mary Williamson,et al. Facebook AI’s WMT20 News Translation Task Submission , 2020, WMT.

[78] Andy Way,et al. Attaining the Unattainable? Reassessing Claims of Human Parity in Neural Machine Translation , 2018, WMT.

[79] Antonio Toral,et al. A Set of Recommendations for Assessing Human-Machine Parity in Language Translation , 2020, J. Artif. Intell. Res..

[80] Karin M. Verspoor,et al. Findings of the 2016 Conference on Machine Translation , 2016, WMT.

[81] Sadaf Abdul-Rauf,et al. Document Level NMT of Low-Resource Languages with Backtranslation , 2020, WMT@EMNLP.

[82] Rejwanul Haque,et al. The ADAPT System Description for the WMT20 News Translation Task , 2020, WMT@EMNLP.

[83] Ulrich Germann. The University of Edinburgh's submission to the German-to-English and English-to-German Tracks in the WMT 2020 News Translation and Zero-shot Translation Robustness Tasks , 2020, WMT@EMNLP.

[84] Patrick Littell,et al. NRC Systems for the 2020 Inuktitut-English News Translation Task , 2020, WMT@EMNLP.

[85] Philipp Koehn,et al. Further Meta-Evaluation of Machine Translation , 2008, WMT@ACL.

[86] Hossein Hassani,et al. Kurdish Interdialect Machine Translation , 2017, VarDial.

[87] Michal Novák,et al. SAO WMT19 Test Suite: Machine Translation of Audit Reports , 2019, WMT.

[88] George Awad,et al. Evaluation of automatic video captioning using direct assessment , 2017, PloS one.

[89] Manish Shrivastava,et al. A3-108 Machine Translation System for Similar Language Translation Shared Task 2020 , 2020, WMT@EMNLP.

[90] Jörg Tiedemann,et al. The MUCOW word sense disambiguation test suite at WMT 2020 , 2020, WMT@EMNLP.

[91] Andreas Eisele,et al. eTranslation's Submissions to the WMT 2020 News Translation Task , 2020, WMT@EMNLP.

[92] Holger Schwenk,et al. WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia , 2019, EACL.

[93] Tomaz Erjavec,et al. hrWaC and slWac: Compiling Web Corpora for Croatian and Slovene , 2011, TSD.

[94] Peng Xu,et al. Attention-Informed Mixed-Language Training for Zero-shot Cross-lingual Task-oriented Dialogue Systems , 2019, AAAI.

[95] Markus Freitag,et al. BLEU Might Be Guilty but References Are Not Innocent , 2020, EMNLP.

[96] Ondrej Bojar,et al. Training Tips for the Transformer Model , 2018, Prague Bull. Math. Linguistics.

[97] Marta R. Costa-jussà,et al. Multilingual Neural Machine Translation: Case-study for Catalan, Spanish and Portuguese Romance Languages , 2020, WMT.

[98] Alberto Poncelas,et al. Neural Machine Translation for translating into Croatian and Serbian , 2020, VARDIAL.

[99] Gabriel Stanovsky,et al. Gender Coreference and Bias Evaluation at WMT 2020 , 2020, WMT@EMNLP.

[100] Hans Uszkoreit,et al. Fine-grained evaluation of German-English Machine Translation based on a Test Suite , 2018, WMT.

[101] Ralph Weischedel,et al. A STUDY OF TRANSLATION ERROR RATE WITH TARGETED HUMAN ANNOTATION , 2005 .

[102] Amanpreet Singh,et al. Infosys Machine Translation System for WMT20 Similar Language Translation Task , 2020, WMT@EMNLP.

[103] Olivier Galibert,et al. Findings of the First Shared Task on Lifelong Learning Machine Translation , 2020, WMT.

[104] Santanu Pal,et al. Neural Machine Translation for Similar Languages: The Case of Indo-Aryan Languages , 2020, WMT.

[105] Philipp Koehn,et al. Findings of the WMT 2020 Shared Task on Parallel Corpus Filtering and Alignment , 2020, WMT.

[106] Philipp Koehn,et al. Findings of the 2009 Workshop on Statistical Machine Translation , 2009, WMT@EACL.

[107] Jeffrey Micher,et al. The Nunavut Hansard Inuktitut–English Parallel Corpus 3.0 with Preliminary Machine Translation Results , 2020, LREC.

[108] Philipp Koehn,et al. Findings of the 2018 Conference on Machine Translation (WMT18) , 2018, WMT.

[109] Xiao Pan,et al. The Volctrans Machine Translation System for WMT20 , 2020, WMT@EMNLP.

[110] Kevin Duh,et al. Automatic Evaluation of Translation Quality for Distant Language Pairs , 2010, EMNLP.

[111] Myle Ott,et al. Scaling Neural Machine Translation , 2018, WMT.

[112] Antonio Toral,et al. Machine Translation for English–Inuktitut with Segmentation, Data Acquisition and Pre-Training , 2020, WMT.

[113] Muhammad Rafi,et al. Attention Transformer Model for Translation of Similar Languages , 2020, WMT.

[114] András Kornai,et al. Parallel corpora for medium density languages , 2007 .

[115] Jun Suzuki,et al. Tohoku-AIP-NTT at WMT 2020 News Translation Task , 2020, WMT@EMNLP.

[116] Ondrej Bojar,et al. Using MT-ComparEval , 2016 .

[117] Jörg Tiedemann,et al. The MuCoW Test Suite at WMT 2019: Automatically Harvested Multilingual Contrastive Word Sense Disambiguation Test Sets for Machine Translation , 2019, WMT.

[118] Hai Zhao,et al. SJTU-NICT's Supervised and Unsupervised Neural Machine Translation Systems for the WMT20 News Translation Task , 2020, WMT@EMNLP.

[119] Ariel Xv,et al. Russian-English Bidirectional Machine Translation System , 2020, WMT.

[120] Ondřej Bojar,et al. WMT20 Document-Level Markable Error Exploration , 2020, WMT@EMNLP.

[121] Daniel Jurafsky,et al. JESC: Japanese-English Subtitle Corpus , 2017, LREC.