Multi-Document Scientific Summarization from a Knowledge Graph-Centric View

Multi-Document Scientific Summarization (MDSS) aims to produce coherent and concise summaries for clusters of topic-relevant scientific papers. This task requires precise understanding of paper content and accurate modeling of cross-paper relationships. Knowledge graphs convey compact and interpretable structured information for documents, which makes them ideal for content modeling and relationship modeling. In this paper, we present KGSum, an MDSS model centred on knowledge graphs during both the encoding and decoding process. Specifically, in the encoding process, two graph-based modules are proposed to incorporate knowledge graph information into paper encoding, while in the decoding process, we propose a two-stage decoder by first generating knowledge graph information of summary in the form of descriptive sentences, followed by generating the final summary. Empirical results show that the proposed architecture brings substantial improvements over baselines on the Multi-Xscience dataset.

[1]  Regina Barzilay,et al.  Generating Related Work , 2021, ArXiv.

[2]  Noah A. Smith,et al.  Explaining Relationships Between Scientific Documents , 2020, ACL.

[3]  Dongyan Zhao,et al.  Capturing Relations between Scientific Papers: An Abstractive Model for Related Work Section Generation , 2021, ACL.

[4]  Yue Dong,et al.  Multi-XScience: A Large-scale Dataset for Extreme Multi-document Summarization of Scientific Articles , 2020, EMNLP.

[5]  Xiaojun Wan,et al.  Multi-Granularity Interaction Network for Extractive and Abstractive Multi-Document Summarization , 2020, ACL.

[6]  Xinyan Xiao,et al.  Leveraging Graph to Improve Abstractive Multi-Document Summarization , 2020, ACL.

[7]  Pengfei Liu,et al.  Heterogeneous Graph Neural Networks for Extractive Document Summarization , 2020, ACL.

[8]  Peter J. Liu,et al.  PEGASUS: Pre-training with Extracted Gap-sentences for Abstractive Summarization , 2019, ICML.

[9]  Omer Levy,et al.  BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension , 2019, ACL.

[10]  Hannaneh Hajishirzi,et al.  Entity, Relation, and Event Extraction with Contextualized Span Representations , 2019, EMNLP.

[11]  Mirella Lapata,et al.  Text Summarization with Pretrained Encoders , 2019, EMNLP.

[12]  Yang Feng,et al.  Incremental Transformer with Deliberation Decoder for Document Grounded Conversations , 2019, ACL.

[13]  Dragomir R. Radev,et al.  Multi-News: A Large-Scale Multi-Document Summarization Dataset and Abstractive Hierarchical Model , 2019, ACL.

[14]  Mirella Lapata,et al.  Hierarchical Transformers for Multi-Document Summarization , 2019, ACL.

[15]  Heyan Huang,et al.  HSDS: An Abstractive Model for Automatic Survey Generation , 2019, DASFAA.

[16]  Mirella Lapata,et al.  Text Generation from Knowledge Graphs with Graph Transformers , 2019, NAACL.

[17]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[18]  Zheng Gao,et al.  Neural Related Work Summarization with a Joint Context-driven Attention Mechanism , 2019, EMNLP.

[19]  Nenghai Yu,et al.  Deliberation Networks: Sequence Generation Beyond One-Pass Decoding , 2017, NIPS.

[20]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[21]  Christopher D. Manning,et al.  Get To The Point: Summarization with Pointer-Generator Networks , 2017, ACL.

[22]  Chao Wu,et al.  KeyphraseDS: Automatic generation of survey by exploiting keyphrase information , 2017, Neurocomputing.

[23]  Yang Liu,et al.  Modeling Coverage for Neural Machine Translation , 2016, ACL.

[24]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Dragomir R. Radev,et al.  Surveyor: A System for Generating Coherent Survey Articles for Scientific Topics , 2015, AAAI.

[26]  Dragomir R. Radev,et al.  Content Models for Survey Generation: A Factoid-Based Evaluation , 2015, ACL.

[27]  Xiaojun Wan,et al.  Automatic Generation of Related Work Sections in Scientific Papers: An Optimization Approach , 2014, EMNLP.

[28]  Min-Yen Kan,et al.  Towards Automated Related Work Summarization , 2010, COLING.

[29]  Nick Cramer,et al.  Automatic Keyword Extraction from Individual Documents , 2010 .

[30]  Dragomir R. Radev,et al.  Using Citations to Generate surveys of Scientific Paradigms , 2009, NAACL.

[31]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[32]  Rada Mihalcea,et al.  TextRank: Bringing Order into Text , 2004, EMNLP.

[33]  Dragomir R. Radev,et al.  LexRank: Graph-based Lexical Centrality as Salience in Text Summarization , 2004, J. Artif. Intell. Res..

[34]  F. W. Levi,et al.  Finite geometrical systems , 1942 .