Unsupervised Dual-Cascade Learning with Pseudo-Feedback Distillation for Query-based Extractive Summarization

We propose Dual-CES – a novel unsupervised, query-focused, multi-document extractive summarizer. Dual-CES builds on top of the Cross Entropy Summarizer (CES) and is designed to better handle the tradeoff between saliency and focus in summarization. To this end, Dual-CES employs a two-step dual-cascade optimization approach with saliency-based pseudo-feedback distillation. Overall, Dual-CES significantly outperforms all other state-of-the-art unsupervised alternatives. Dual-CES is even shown to be able to outperform strong supervised summarizers.

[1]  David Konopnicki,et al.  Unsupervised Query-Focused Multi-Document Summarization using the Cross Entropy Method , 2017, SIGIR.

[2]  Yu-lin He,et al.  OWA operator based link prediction ensemble for social network , 2015, Expert Syst. Appl..

[3]  Jade Goldstein-Stewart,et al.  The use of MMR, diversity-based reranking for reordering documents and producing summaries , 1998, SIGIR '98.

[4]  John D. Lafferty,et al.  A risk minimization framework for information retrieval , 2006, Inf. Process. Manag..

[5]  Dirk P. Kroese,et al.  The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation and Machine Learning , 2004 .

[6]  Lih-Yuan Deng,et al.  The Cross-Entropy Method: A Unified Approach to Combinatorial Optimization, Monte-Carlo Simulation, and Machine Learning , 2006, Technometrics.

[7]  Hang Li,et al.  Reader-Aware Multi-Document Summarization via Sparse Coding , 2015, IJCAI.

[8]  Zhi-Hong Deng,et al.  An Unsupervised Multi-Document Summarization Framework Based on Neural Document Model , 2016, COLING.

[9]  Yen-Chun Chen,et al.  Fast Abstractive Summarization with Reinforce-Selected Sentence Rewriting , 2018, ACL.

[10]  Lucy Vanderwende,et al.  Exploring Content Models for Multi-Document Summarization , 2009, NAACL.

[11]  M. de Rijke,et al.  Leveraging Contextual Sentence Relations for Extractive Summarization Using a Neural Attention Model , 2017, SIGIR.

[12]  Hui Lin,et al.  A Class of Submodular Functions for Document Summarization , 2011, ACL.

[13]  Vishal Gupta,et al.  Recent automatic text summarization techniques: a survey , 2016, Artificial Intelligence Review.

[14]  Jade Goldstein-Stewart,et al.  The Use of MMR, Diversity-Based Reranking for Reordering Documents and Producing Summaries , 1998, SIGIR Forum.

[15]  Hoa Trang Dang,et al.  Overview of DUC 2005 , 2005 .

[16]  M. de Rijke,et al.  Sentence Relations for Extractive Summarization with Deep Neural Networks , 2018, ACM Trans. Inf. Syst..

[17]  Jing Long,et al.  Query-oriented unsupervised multi-document summarization via deep learning model , 2015, Expert Syst. Appl..

[18]  David Konopnicki,et al.  A Summarization System for Scientific Documents , 2019, EMNLP.

[19]  W. Bruce Croft,et al.  Relevance-Based Language Models , 2001, SIGIR '01.

[20]  Chris H. Q. Ding,et al.  Integrating Clustering and Multi-Document Summarization by Bi-Mixture Probabilistic Latent Semantic Analysis (PLSA) with Sentence Bases , 2011, AAAI.

[21]  Chun Chen,et al.  Document Summarization Based on Data Reconstruction , 2012, AAAI.

[22]  David Konopnicki,et al.  An Editorial Network for Enhanced Document Summarization , 2019, EMNLP.

[23]  Richard Socher,et al.  A Deep Reinforced Model for Abstractive Summarization , 2017, ICLR.

[24]  Barbara Plank,et al.  Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies , 2011 .

[25]  Xiaojun Wan,et al.  Compressive Document Summarization via Sparse Optimization , 2015, IJCAI.

[26]  Furu Wei,et al.  AttSum: Joint Learning of Focusing and Summarization with Neural Attention , 2016, COLING.

[27]  Yang Xu,et al.  Query dependent pseudo-relevance feedback based on wikipedia , 2009, SIGIR.

[28]  Hang Li,et al.  Cascaded Attention based Unsupervised Information Distillation for Compressive Summarization , 2017, EMNLP.

[29]  Piji Li,et al.  Salience Estimation via Variational Auto-Encoders for Multi-Document Summarization , 2017, AAAI.

[30]  Mark T. Maybury,et al.  Automatic Summarization , 2002, Computational Linguistics.

[31]  Dirk P. Kroese,et al.  Generalized Cross-entropy Methods with Applications to Rare-event Simulation and Optimization , 2007, Simul..

[32]  Mirella Lapata,et al.  Ranking Sentences for Extractive Summarization with Reinforcement Learning , 2018, NAACL.

[33]  Qin Lu,et al.  Applying regression models to query-focused multi-document summarization , 2011, Inf. Process. Manag..

[34]  Dilek Z. Hakkani-Tür,et al.  A Hybrid Hierarchical Model for Multi-Document Summarization , 2010, ACL.

[35]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[36]  Xiaojun Wan,et al.  CTSUM: extracting more certain summaries for news articles , 2014, SIGIR.