Paraphrase Generation with Collaboration between the Forward and the Backward Decoder

Automatic paraphrase generation plays a key role in many natural language applications. The dominant paraphrase generation models are the encoder-decoder neural networks with attention, where the decoder uses the information of the source text while predicting target text. However, the outputs of these paraphrase models often suffer the semantic error problem. This problem is caused by the inadequate information of the decoder. In this work, we introduce a novel neural model to solve this problem, called Collaboration between the Forward and the Backward Decoder. Specifically, the hidden states of the backward decoder are used as supplementary information of the forward decoder. Therefore, the forward decoder can generate more reasonable paraphrase text using the target-side future contextual. Conversely, the backward decoder employs the hidden states of the forward decoder to prevent the semantic error problem. As two experimental examples show, the proposed model can generate the high-quality paraphrase through this collaboration mechanism. The empirical study on two benchmark datasets demonstrates that our model outperforms some baselines and achieves the state-of-the-art performance.