A novel ILP framework for summarizing content with high lexical variety

Summarizing content contributed by individuals can be challenging, because people make different lexical choices even when describing the same events. However, there remains a significant need to summarize such content. Examples include the student responses to post-class reflective questions, product reviews, and news articles published by different news agencies related to the same events. High lexical diversity of these documents hinders the system's ability to effectively identify salient content and reduce summary redundancy. In this paper, we overcome this issue by introducing an integer linear programming-based summarization framework. It incorporates a low-rank approximation to the sentence-word co-occurrence matrix to intrinsically group semantically-similar lexical items. We conduct extensive experiments on datasets of student responses, product reviews, and news documents. Our approach compares favorably to a number of extractive baselines as well as a neural abstractive summarization system. The paper finally sheds light on when and why the proposed framework is effective at summarizing content with high lexical variety.

[1]  Fei Liu,et al.  An Improved Phrase-based Approach to Annotating and Summarizing Student Course Responses , 2016, COLING.

[2]  Mor Naaman,et al.  Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies , 2018, NAACL.

[3]  Noah A. Smith,et al.  Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2016, ACL 2016.

[4]  Nazli Goharian,et al.  Revisiting Summarization Evaluation for Scientific Articles , 2016, LREC.

[5]  Christopher D. Manning,et al.  Get To The Point: Summarization with Pointer-Generator Networks , 2017, ACL.

[6]  Bowen Zhou,et al.  Sequence-to-Sequence RNNs for Text Summarization , 2016, ArXiv.

[7]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[8]  Diane J. Litman,et al.  Empirical analysis of exploiting review helpfulness for extractive summarization of online reviews , 2014, COLING.

[9]  Phil Blunsom,et al.  Teaching Machines to Read and Comprehend , 2015, NIPS.

[10]  Benoît Favre,et al.  Concept-based Summarization using Integer Linear Programming: From Concept Pruning to Multiple Optimal Solutions , 2015, EMNLP.

[11]  Vasile Rus,et al.  SEMILAR: The Semantic Similarity Toolkit , 2013, ACL.

[12]  Kien A. Hua,et al.  Toward Extractive Summarization of Online Forum Discussions via Hierarchical Attention Networks , 2018, FLAIRS.

[13]  Xiaojun Wan,et al.  Abstractive Document Summarization with a Graph-Based Attentional Neural Model , 2017, ACL.

[14]  W. Harwood.  The One-Minute Paper:: A Communication Tool for Large Lecture Classes , 1996 .

[15]  Graham Neubig,et al.  Controlling Output Length in Neural Encoder-Decoders , 2016, EMNLP.

[16]  Robert Tibshirani,et al.  Spectral Regularization Algorithms for Learning Large Incomplete Matrices , 2010, J. Mach. Learn. Res..

[17]  Franck Dernoncourt,et al.  A Discourse-Aware Attention Model for Abstractive Summarization of Long Documents , 2018, NAACL.

[18]  Furu Wei,et al.  Faithful to the Original: Fact Aware Neural Abstractive Summarization , 2017, AAAI.

[19]  Rui Zhang,et al.  Graph-based Neural Multi-Document Summarization , 2017, CoNLL.

[20]  Dan Klein,et al.  Learning-Based Single-Document Summarization with Compression and Anaphoricity Constraints , 2016, ACL.

[21]  Marc Moens,et al.  Articles Summarizing Scientific Articles: Experiments with Relevance and Rhetorical Status , 2002, CL.

[22]  Jingtao Wang,et al.  CourseMIRROR: Enhancing Large Classroom Instructor-Student Interactions via Mobile Interfaces and Natural Language Processing , 2015, CHI Extended Abstracts.

[23]  Jade Goldstein-Stewart,et al.  The use of MMR, diversity-based reranking for reordering documents and producing summaries , 1998, SIGIR '98.

[24]  Ming Zhou,et al.  A Redundancy-Aware Sentence Regression Framework for Extractive Summarization , 2016, COLING.

[25]  Lin Zhao,et al.  Improving Multi-documents Summarization by Sentence Compression based on Expanded Constituent Parse Trees , 2014, EMNLP.

[26]  Benoit Favre,et al.  A Scalable Global Model for Summarization , 2009, ILP 2009.

[27]  Ani Nenkova,et al.  Automatic Summarization , 2011, ACL.

[28]  Chun Chen,et al.  Document Summarization Based on Data Reconstruction , 2012, AAAI.

[29]  Yi-Kai Liu,et al.  Multilingual Summarization: Dimensionality Reduction and a Step Towards Optimal Term Coverage , 2013 .

[30]  André F. T. Martins,et al.  Fast and Robust Compressive Summarization with Dual Decomposition and Multi-Task Learning , 2013, ACL.

[31]  Dimitra Gkatzia,et al.  Generating Student Feedback from Time-Series Data Using Reinforcement Learning , 2013, ENLG.

[32]  Jason Weston,et al.  A Neural Attention Model for Abstractive Sentence Summarization , 2015, EMNLP.

[33]  Dragomir R. Radev,et al.  Generating Extractive Summaries of Scientific Paradigms , 2013, J. Artif. Intell. Res..

[34]  Carolyn Penstein Rosé,et al.  Shared Task on Prediction of Dropout Over Time in Massively Open Online Courses , 2014, EMNLP 2014.

[35]  Giuseppe Carenini,et al.  Abstractive Summarization of Product Reviews Using Discourse Structure , 2014, EMNLP.

[36]  Noah A. Smith,et al.  Summarization with a Joint Model for Sentence Extraction and Compression , 2009, ILP 2009.

[37]  Fei Liu,et al.  Document Summarization via Guided Sentence Compression , 2013, EMNLP.

[38]  Ion Androutsopoulos,et al.  Extractive Multi-Document Summarization with Integer Linear Programming and Support Vector Regression , 2012, COLING.

[39]  Jade Goldstein-Stewart,et al.  The use of MMR, diversity-based reranking for reordering documents and producing summaries , 1998, SIGIR '98.

[40]  Sun Park,et al.  Automatic generic document summarization based on non-negative matrix factorization , 2009, Inf. Process. Manag..

[41]  Regina Barzilay,et al.  Information Fusion in the Context of Multi-Document Summarization , 1999, ACL.

[42]  Jingtao Wang,et al.  Scaling Reflection Prompts in Large Classrooms via Mobile Interfaces and Natural Language Processing , 2017, IUI.

[43]  Dragomir R. Radev,et al.  A Low-Rank Approximation Approach to Learning Joint Embeddings of News Stories and Images for Timeline Summarization , 2016, HLT-NAACL.

[44]  Masaaki Nagata,et al.  Cutting-off Redundant Repeating Generations for Neural Abstractive Summarization , 2016, EACL.

[45]  Yvette Graham,et al.  Re-evaluating Automatic Summarization with BLEU and 192 Shades of ROUGE , 2015, EMNLP.

[46]  San Cristóbal Mateo,et al.  The Lack of A Priori Distinctions Between Learning Algorithms , 1996 .

[47]  Carolyn Penstein Rosé,et al.  Sentiment Analysis in MOOC Discussion Forums: What does it tell us? , 2014, EDM.

[48]  Ramakanth Pasunuru,et al.  Soft Layer-Specific Multi-Task Summarization with Entailment and Question Generation , 2018, ACL.

[49]  John M. Conroy,et al.  Vector Space Models for Scientific Document Summarization , 2015, VS@HLT-NAACL.

[50]  Nazli Goharian,et al.  Scientific Article Summarization Using Citation-Context and Article’s Discourse Structure , 2015, EMNLP.

[51]  Ben Taskar,et al.  Determinantal Point Processes for Machine Learning , 2012, Found. Trends Mach. Learn..

[52]  R. C. Wilson.  Improving Faculty Teaching: Effective Use of Student Evaluations and Consultants. , 1986 .

[53]  Mirella Lapata,et al.  Ranking Sentences for Extractive Summarization with Reinforcement Learning , 2018, NAACL.

[54]  Naoaki Okazaki,et al.  Neural Headline Generation on Abstract Meaning Representation , 2016, EMNLP.

[55]  Fred Paas,et al.  Reflection prompts and tutor feedback in a web-based learning environment: effects on students' self-regulated learning competence , 2004, Comput. Hum. Behav..

[56]  Yejin Choi,et al.  Deep Communicating Agents for Abstractive Summarization , 2018, NAACL.

[57]  Kathleen McKeown,et al.  The decomposition of human-written summary sentences , 1999, SIGIR '99.

[58]  Seungwhan Moon,et al.  Identifying Student Leaders from MOOC Discussion Forums through Language Influence , 2014, EMNLP 2014.

[59]  Hoa Trang Dang,et al.  Overview of the TAC 2008 Update Summarization Task , 2008, TAC.

[60]  W. Bruce Croft,et al.  Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , 2013 .

[61]  Dilek Z. Hakkani-Tür,et al.  The ICSI Summarization System at TAC 2008 , 2008, TAC.

[62]  Hui Lin,et al.  Multi-document Summarization via Budgeted Maximization of Submodular Functions , 2010, NAACL.

[63]  Fei Liu,et al.  Abstract Meaning Representation for Multi-Document Summarization , 2018, COLING.

[64]  Dragomir R. Radev,et al.  Centroid-based summarization of multiple documents , 2004, Inf. Process. Manag..

[65]  Lin Zhao,et al.  Using External Resources and Joint Learning for Bigram Weighting in ILP-Based Multi-Document Summarization , 2015, NAACL.

[66]  Marc'Aurelio Ranzato,et al.  Sequence Level Training with Recurrent Neural Networks , 2015, ICLR.

[67]  Ani Nenkova,et al.  Beyond SumBasic: Task-focused summarization with sentence simplification and lexical expansion , 2007, Information Processing & Management.

[68]  Yang Liu,et al.  Fast Joint Compression and Summarization via Graph Cuts , 2013, EMNLP.

[69]  Stephen Krause,et al.  The effectiveness of students' daily reflections on learning in engineering context , 2011 .

[70]  Wenting Xiong.  Helpfulness-Guided Review Summarization , 2013, HLT-NAACL.

[71]  Fei Liu,et al.  Automatic Summarization of Student Course Feedback , 2016, HLT-NAACL.

[72]  Richard Socher,et al.  A Deep Reinforced Model for Abstractive Summarization , 2017, ICLR.

[73]  Hui Lin,et al.  A Repository of State of the Art and Competitive Baseline Summaries for Generic News Summarization , 2014, LREC.

[74]  Fei Liu,et al.  Towards Abstractive Speech Summarization: Exploring Unsupervised and Supervised Approaches for Spoken Utterance Compression , 2013, IEEE Transactions on Audio, Speech, and Language Processing.

[75]  Dan Klein,et al.  Jointly Learning to Extract and Compress , 2011, ACL.

[76]  Xun Wang,et al.  Exploring Text Links for Coherent Multi-Document Summarization , 2016, COLING.

[77]  Ming Zhou,et al.  Selective Encoding for Abstractive Sentence Summarization , 2017, ACL.

[78]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[79]  Jade Goldstein-Stewart,et al.  The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries , 1998, SIGIR Forum.

[80]  Lin Zhao,et al.  Structure-Infused Copy Mechanisms for Abstractive Summarization , 2018, COLING.

[81]  Nazli Goharian,et al.  Scientific document summarization via citation contextualization and scientific discourse , 2017, International Journal on Digital Libraries.

[82]  Bing Liu,et al.  Opinion spam and analysis , 2008, WSDM '08.

[83]  Kwangsu Cho.  Machine Classification of Peer Comments in Physics , 2008, EDM.

[84]  Peter A. Rankel.  Statistical Analysis of Text Summarization Evaluation , 2016 .

[85]  Zhen-Hua Ling,et al.  Distraction-Based Neural Networks for Document Summarization , 2016, ArXiv.

[86]  D. Boud,et al.  Reflection, turning experience into learning , 1985 .

[87]  Chris H. Q. Ding,et al.  Multi-document summarization via sentence-level semantic analysis and symmetric matrix factorization , 2008, SIGIR '08.

[88]  Frederick Mosteller,et al.  The "Muddiest Point in the Lecture" as a feedback device , 1989 .

[89]  Carolyn Penstein Rosé,et al.  Linguistic Reflections of Student Engagement in Massive Open Online Courses , 2014, ICWSM.

[90]  Diane J. Litman,et al.  Summarizing Student Responses to Reflection Prompts , 2015, EMNLP.

[91]  Omer Levy,et al.  word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method , 2014, ArXiv.

[92]  Dragomir R. Radev,et al.  LexRank: Graph-based Lexical Centrality as Salience in Text Summarization , 2004, J. Artif. Intell. Res..

[93]  Jingtao Wang,et al.  Enhancing Instructor-Student and Student-Student Interactions with Mobile Interfaces and Summarization , 2015, NAACL.