Joint Optimization of User-desired Content in Multi-document Summaries by Learning from User Feedback

In this paper, we propose an extractive multi-document summarization (MDS) system using joint optimization and active learning for content selection grounded in user feedback. Our method interactively obtains user feedback to gradually improve the results of a state-of-the-art integer linear programming (ILP) framework for MDS. Our methods complement fully automatic methods in producing high-quality summaries with a minimum number of iterations and feedbacks. We conduct multiple simulation-based experiments and analyze the effect of feedback-based concept selection in the ILP setup in order to maximize the user-desired content in the summary.

[1]  Hans Peter Luhn,et al.  The Automatic Creation of Literature Abstracts , 1958, IBM J. Res. Dev..

[2]  Rada Mihalcea,et al.  TextRank: Bringing Order into Text , 2004, EMNLP.

[3]  Francisco Casacuberta,et al.  Active learning for interactive machine translation , 2012, EACL.

[4]  Ryan T. McDonald.  A Study of Global Inference Algorithms in Multi-document Summarization , 2007, ECIR.

[5]  Dan Klein,et al.  Accurate Unlexicalized Parsing , 2003, ACL.

[6]  Iryna Gurevych,et al.  GermEval-2014: Nested Named Entity Recognition with Neural Networks , 2014 .

[7]  Pablo Gervás,et al.  User-model based personalized summarization , 2007, Inf. Process. Manag..

[8]  Philipp Koehn,et al.  Neural Interactive Translation Prediction , 2016, AMTA.

[9]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL 2006.

[10]  Mirella Lapata,et al.  Multiple Aspect Summarization Using Integer Linear Programming , 2012, EMNLP.

[11]  Benoit Favre,et al.  A Scalable Global Model for Summarization , 2009, ILP 2009.

[12]  Benoît Favre,et al.  Concept-based Summarization using Integer Linear Programming: From Concept Pruning to Multiple Optimal Solutions , 2015, EMNLP.

[13]  Helen Petrie,et al.  The Evaluation of Accessibility, Usability, and User Experience , 2009, The Universal Access Handbook.

[14]  Dong-Hong Ji,et al.  Context-Enhanced Personalized Social Summarization , 2012, COLING.

[15]  John Platt,et al.  Probabilistic Outputs for Support vector Machines and Comparisons to Regularized Likelihood Methods , 1999 .

[16]  Lucy Vanderwende,et al.  Exploring Content Models for Multi-Document Summarization , 2009, NAACL.

[17]  Sun Park,et al.  Automatic query-based personalized summarization that uses pseudo relevance feedback with NMF , 2010, ICUIMC '10.

[18]  Takehito Utsuro,et al.  A Web-based English Abstract Writing Tool Using a Tagged E-J Parallel Corpus , 2002, LREC.

[19]  Ingrid Zukerman,et al.  Aspect-Based Personalized Text Summarization , 2008, AH.

[20]  Klaus Zechner,et al.  Automatic Summarization of Open-Domain Multiparty Dialogues in Diverse Genres , 2002, CL.

[21]  Xin Liu,et al.  Generic text summarization using relevance measure and latent semantic analysis , 2001, SIGIR '01.

[22]  Christian Igel,et al.  Active learning with support vector machines , 2014, WIREs Data Mining Knowl. Discov..

[23]  Iryna Gurevych,et al.  Bridging the gap between extractive and abstractive summaries: Creation and evaluation of coherent extracts from heterogeneous sources , 2016, COLING.

[24]  Dragomir R. Radev,et al.  LexRank: Graph-based Lexical Centrality as Salience in Text Summarization , 2004, J. Artif. Intell. Res..

[25]  Timothy C. Craven.  Abstracts produced using computer assistance , 2000, J. Am. Soc. Inf. Sci..

[26]  Yang Liu,et al.  Using Supervised Bigram-based ILP for Extractive Summarization , 2013, ACL.

[27]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[28]  John M. Conroy,et al.  An Assessment of the Accuracy of Automatic Evaluation in Summarization , 2012, EvalMetrics@NAACL-HLT.

[29]  Constantin Orasan,et al.  CAST: A computer-aided summarisation tool , 2003, EACL.

[30]  Constantin Orasan,et al.  Computer-aided summarisation – what the user really wants , 2006, LREC.

[31]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[32]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL.

[33]  Wei-Ying Ma,et al.  A Study for Document Summarization Based on Personal Annotation , 2003, HLT-NAACL 2003.

[34]  Hui Lin,et al.  A Repository of State of the Art and Competitive Baseline Summaries for Generic News Summarization , 2014, LREC.

[35]  Ming Zhou,et al.  TGSum: Build Tweet Guided Multi-Document Summarization Dataset , 2015, AAAI.

[36]  Dan Klein,et al.  Jointly Learning to Extract and Compress , 2011, ACL.

[37]  Patrick Gallinari,et al.  Extended Recommendation Framework: Generating the Text of a User Review as a Personalized Summary , 2015, CBRecSys@RecSys.