Improving neural machine translation with sentence alignment learning

Abstract Neural machine translation (NMT) optimized by maximum likelihood estimation (MLE) usually lacks the guarantee of translation adequacy. To alleviate this problem, we propose an NMT approach that heightens the adequacy in machine translation by transferring the semantic knowledge from bilingual sentence alignment learning. Specifically, we first design a discriminator that learns to estimate sentence aligning score over translation candidates. The discriminator is constructed by gated self-attention based sentence encoders and trained with an N-pair loss for better capturing lexical evidences from bilingual sentence pairs. Then we propose an adversarial training framework as well as a sentence alignment-aware decoding method for NMT to transfer the discriminator’s learned semantic knowledge to NMT models. We conduct our experiments on Chinese → English, Uyghur → Chinese and English → German translation tasks. Experimental results show that our proposed methods outperform baseline NMT models on all these three translation tasks. Further analysis also indicates the characteristics of our approaches and details the semantic knowledge that transfered from the discriminator to the NMT model.