A Combined Extractive With Abstractive Model for Summarization

Aiming at the difficulties in document-level summarization, this paper presents a two-stage, extractive and then abstractive summarization model. In the first stage, we extract the important sentences by combining sentences similarity matrix (only used for the first time) or pseudo-title, which takes full account of the features (such as sentence position, paragraph position, and more.). To extract coarse-grained sentences from a document, and considers the sentence differentiation for the most important sentences in the document. The second stage is abstractive, and we use beam search algorithm to restructure and rewrite these syntactic blocks of these extracted sentences. Newly generated summary sentence serves as the pseudo-summary of the next round. Globally optimal pseudo-title acts as the final summarization. Extensive experiments have been performed on the corresponding data set, and the results show our model can obtain better results.

[1]  Ming Zhou,et al.  Selective Encoding for Abstractive Sentence Summarization , 2017, ACL.

[2]  Yi-Kai Liu,et al.  Multilingual Summarization: Dimensionality Reduction and a Step Towards Optimal Term Coverage , 2013 .

[3]  Guangzhen Zhao,et al.  Language Model-Driven Topic Clustering and Summarization for News Articles , 2019, IEEE Access.

[4]  J.N. Madhuri,et al.  Extractive Text Summarization Using Sentence Ranking , 2019, 2019 International Conference on Data Science and Communication (IconDSC).

[5]  Mirella Lapata,et al.  Discourse Constraints for Document Compression , 2010, CL.

[6]  Alexander M. Rush,et al.  Bottom-Up Abstractive Summarization , 2018, EMNLP.

[7]  Rada Mihalcea,et al.  TextRank: Bringing Order into Text , 2004, EMNLP.

[8]  Piji Li,et al.  Deep Recurrent Generative Decoder for Abstractive Text Summarization , 2017, EMNLP.

[9]  Navdeep Jaitly,et al.  Pointer Networks , 2015, NIPS.

[10]  Taner UÇKAN,et al.  Graph-Based Suggestion For Text Summarization , 2018, 2018 International Conference on Artificial Intelligence and Data Processing (IDAP).

[11]  Marc'Aurelio Ranzato,et al.  Sequence Level Training with Recurrent Neural Networks , 2015, ICLR.

[12]  Qiang Qu,et al.  Exploring Human-Like Reading Strategy for Abstractive Text Summarization , 2019, AAAI.

[13]  Mohammed Al-Dhelaan,et al.  An Approach for Combining Multiple Weighting Schemes and Ranking Methods in Graph-Based Multi-Document Summarization , 2019, IEEE Access.

[14]  Hang Li,et al.  “ Tony ” DNN Embedding for “ Tony ” Selective Read for “ Tony ” ( a ) Attention-based Encoder-Decoder ( RNNSearch ) ( c ) State Update s 4 SourceVocabulary Softmax Prob , 2016 .

[15]  Iryna Gurevych,et al.  A Reinforcement Learning Approach for Adaptive Single- and Multi-Document Summarization , 2015, GSCL.

[16]  Furu Wei,et al.  Retrieve, Rerank and Rewrite: Soft Template Based Neural Summarization , 2018, ACL.

[17]  Eliseo Reategui,et al.  Using a Text Mining Tool to Support Text Summarization , 2012, 2012 IEEE 12th International Conference on Advanced Learning Technologies.

[18]  Keith C. C. Chan,et al.  Learning Latent Factors for Community Identification and Summarization , 2018, IEEE Access.

[19]  Jinlong Li,et al.  Keyphrase Enhanced Diverse Beam Search: A Content-Introducing Approach to Neural Text Generation , 2019, IEEE Access.

[20]  Ming Zhou,et al.  S-Net: From Answer Extraction to Answer Generation for Machine Reading Comprehension , 2017, AAAI 2017.

[21]  Christopher D. Manning,et al.  Get To The Point: Summarization with Pointer-Generator Networks , 2017, ACL.

[22]  Lukasz Kaiser,et al.  Sentence Compression by Deletion with LSTMs , 2015, EMNLP.

[23]  Huifang Ma,et al.  Text Summarization Method Based on Double Attention Pointer Network , 2020, IEEE Access.

[24]  Bowen Zhou,et al.  SummaRuNNer: A Recurrent Neural Network Based Sequence Model for Extractive Summarization of Documents , 2016, AAAI.

[25]  Heechan Kim,et al.  A Context based Coverage Model for Abstractive Document Summarization , 2019, 2019 International Conference on Information and Communication Technology Convergence (ICTC).

[26]  Yu Shanshan,et al.  Improved TextRank-based Method for Automatic Summarization , 2016 .

[27]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[28]  Yang Liu,et al.  Fine-tune BERT for Extractive Summarization , 2019, ArXiv.

[29]  Bowen Zhou,et al.  Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond , 2016, CoNLL.

[30]  Mohamad Abdolahi,et al.  An overview on extractive text summarization , 2017, 2017 IEEE 4th International Conference on Knowledge-Based Engineering and Innovation (KBEI).

[31]  Zhen-Hua Ling,et al.  Distraction-Based Neural Networks for Modeling Document , 2016, IJCAI.

[32]  Rakesh M. Verma,et al.  Combining Syntax and Semantics for Automatic Extractive Single-Document Summarization , 2012, CICLing.

[33]  Joelle Pineau,et al.  An Actor-Critic Algorithm for Sequence Prediction , 2016, ICLR.

[34]  Dejun Mu,et al.  Word-sentence co-ranking for automatic extractive text summarization , 2017, Expert Syst. Appl..

[35]  Richard Socher,et al.  A Deep Reinforced Model for Abstractive Summarization , 2017, ICLR.

[36]  Xiaojun Wan,et al.  Abstractive Document Summarization with a Graph-Based Attentional Neural Model , 2017, ACL.

[37]  Christina Lioma,et al.  Graph-based term weighting for information retrieval , 2011, Information Retrieval.

[38]  Jason Weston,et al.  A Neural Attention Model for Abstractive Sentence Summarization , 2015, EMNLP.