Incorporating Syntactic Knowledge in Neural Quality Estimation for Machine Translation

Translation quality estimation aims at evaluating the machine translation output without references. State-of-the-art quality estimation methods based on neural networks have certain capability of implicitly learning the syntactic information from sentence-aligned parallel corpus. However, they still fail to capture the deep structural syntactic details of the sentences. This paper proposes a method that explicitly incorporates source syntax in neural quality estimation. Specifically, the parse trees of source sentences are linearized, and the sequence labels are combined with the source sequence through hierarchical encoding to obtain a more complete and deeper source encoding vector. The hidden relationships between the source syntactic structure and the translation quality are modeled to discover the syntactic errors in the translation. Experimental results on WMT17 quality estimation datasets show that the sentence-level Pearson correlation score and the word-level F1–mult score can both be improved by the syntactic knowledge.