Large-scale Multitask Learning for Machine Translation Quality Estimation

Multitask learning has been proven a useful technique in a number of Natural Language Processing applications where data is scarce and naturally diverse. Examples include learning from data of different domains and learning from labels provided by multiple annotators. Tasks in these scenarios would be the domains or the annotators. When faced with limited data for each task, a framework for the learning of tasks in parallel while using a shared representation is clearly helpful: what is learned for a given task can be transferred to other tasks while the peculiarities of each task are still modelled. Focusing on machine translation quality estimation as application, in this paper we show that multitask learning is also useful in cases where data is abundant. Based on two large-scale datasets, we explore models with multiple annotators and multiple languages and show that state-of-the-art multitask learning algorithms lead to improved results in all settings.

[1]  Lucia Specia,et al.  QuEst - A translation quality estimation framework , 2013, ACL.

[2]  N. Draper,et al.  Applied Regression Analysis , 1966 .

[3]  Philipp Koehn,et al.  Findings of the 2015 Workshop on Statistical Machine Translation , 2015, WMT@EMNLP.

[4]  Philipp Koehn,et al.  Findings of the 2012 Workshop on Statistical Machine Translation , 2012, WMT@NAACL-HLT.

[5]  Ralph Weischedel,et al.  A STUDY OF TRANSLATION ERROR RATE WITH TARGETED HUMAN ANNOTATION , 2005 .

[6]  Lucia Specia,et al.  Quality estimation for translation selection , 2014, EAMT.

[7]  Hal Daumé,et al.  Frustratingly Easy Domain Adaptation , 2007, ACL.

[8]  Neil D. Lawrence,et al.  Kernels for Vector-Valued Functions: a Review , 2011, Found. Trends Mach. Learn..

[9]  Maarit Koponen,et al.  Comparing human perceptions of post-editing effort with post-editing operations , 2012, WMT@NAACL-HLT.

[10]  Philipp Koehn,et al.  Findings of the 2014 Workshop on Statistical Machine Translation , 2014, WMT@ACL.

[11]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[12]  Claudio Gentile,et al.  Linear Algorithms for Online Multitask Classification , 2010, COLT.

[13]  S. Lewis,et al.  Regression analysis , 2007, Practical Neurology.

[14]  Lucia Specia,et al.  An Investigation on the Effectiveness of Features for Translation Quality Estimation , 2013, MTSUMMIT.

[15]  Zoubin Ghahramani,et al.  Sparse Gaussian Processes using Pseudo-inputs , 2005, NIPS.

[16]  Lucia Specia,et al.  Joint Emotion Analysis via Multi-task Gaussian Processes , 2014, EMNLP.

[17]  Christopher D. Manning,et al.  Hierarchical Bayesian Domain Adaptation , 2009, NAACL.

[18]  Lucia Specia,et al.  SHEF-Lite: When Less is More for Translation Quality Estimation , 2013, WMT@ACL.

[19]  Lucia Specia,et al.  Modelling Annotator Bias with Multi-task Gaussian Processes: An Application to Machine Translation Quality Estimation , 2013, ACL.

[20]  José Guilherme Camargo de Souza,et al.  Machine Translation Quality Estimation Across Domains , 2014, COLING.

[21]  Rich Caruana,et al.  Multitask Learning , 1998, Encyclopedia of Machine Learning and Data Mining.

[22]  Matthew G. Snover,et al.  A Study of Translation Edit Rate with Targeted Human Annotation , 2006, AMTA.

[23]  Edwin V. Bonilla,et al.  Multi-task Gaussian Process Prediction , 2007, NIPS.

[24]  Elisa Ricci,et al.  Online Multitask Learning for Machine Translation Quality Estimation , 2015, ACL.

[25]  Bruno Pouliquen,et al.  Tapta: A user-driven translation system for patent documents based on domain-aware Statistical Machine Translation , 2011, EAMT.

[26]  Marco Turchi,et al.  Towards a combination of online and multitask learning for MT quality estimation: a preliminary study , 2014, AMTA.