Abstractive Summarization with Keyword and Generated Word Attention

Abstractive summarization is a important task in natural language processing field. In previous work, the sequence-to-sequence based models are widely used for abstractive summarization task. However, most of the current abstractive summarization models still suffer from two problems. One is that it is difficult for these models to learn an accurate source contextual representation from the redundancy and noisy source text at each decoding step. Another is the information loss problem, which is ignored in previous work. The inability of these models to effectively exploit previously generated words led to this problem. In order to address these two problems, in this paper, we propose a novel keyword and generated word attention model. Specifically, the proposed model first employs the hidden state of decoder to capture relevant keywords and previously generated words contextual at each time step. The model then utilizes obtained keywords and generated words contextual to create keywords-aware and generated words-aware source contextual, respectively. The keywords contextual contributes to learn an accurate source contextual representation, and the generated words contextual can alleviate the information loss problem. Experimental results on a popular Chinese social media dataset demonstrate that the proposed model outperforms baselines and achieves the state-of-the-art performance.