Query-Oriented Multi-Document Summarization via Unsupervised Deep Learning

Extractive style query oriented multi document summarization generates the summary by extracting a proper set of sentences from multiple documents based on the pre given query. This paper proposes a novel multi document summarization framework via deep learning model. This uniform framework consists of three parts: concepts extraction, summary generation, and reconstruction validation, which work together to achieve the largest coverage of the documents content. A new query oriented extraction technique is proposed to concentrate distributed information to hidden units layer by layer. Then, the whole deep architecture is fine tuned by minimizing the information loss of reconstruction validation. According to the concentrated information, dynamic programming is used to seek most informative set of sentences as the summary. Experiments on three bench mark datasets demonstrate the effectiveness of the proposed framework and algorithms.

[1]  Xiaojun Wan,et al.  Topic analysis for topic-focused multi-document summarization , 2009, CIKM.

[2]  Furu Wei,et al.  A document-sensitive graph model for multi-document summarization , 2010, Knowledge and Information Systems.

[3]  M. F. Porter,et al.  An algorithm for suffix stripping , 1997 .

[4]  Tai Sing Lee,et al.  Hierarchical Bayesian inference in the visual cortex. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[5]  Vasileios Hatzivassiloglou,et al.  A Formal Model for Information Selection in Multi-Sentence Text Extraction , 2004, COLING.

[6]  Tie-Yan Liu,et al.  Learning to rank: from pairwise approach to listwise approach , 2007, ICML '07.

[7]  Geoffrey E. Hinton,et al.  Generating more realistic images using gated MRF's , 2010, NIPS.

[8]  Y. Liu,et al.  Bilinear deep learning for image classification , 2011, ACM Multimedia.

[9]  Khanpour Hamed.  Sentence extraction for summarization and Notetaking , 2009 .

[10]  Yoshua Bengio,et al.  An empirical evaluation of deep architectures on problems with many factors of variation , 2007, ICML '07.

[11]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[12]  Dragomir R. Radev,et al.  Centroid-based summarization of multiple documents , 2004, Inf. Process. Manag..

[13]  Phyllis B. Baxendale,et al.  Machine-Made Index for Technical Literature - An Experiment , 1958, IBM J. Res. Dev..

[14]  Qin Lu,et al.  Applying regression models to query-focused multi-document summarization , 2011, Inf. Process. Manag..

[15]  Xiaoyan Zhu,et al.  A Comparative Study on Ranking and Selection Strategies for Multi-Document Summarization , 2010, COLING.

[16]  R RadevDragomir,et al.  Centroid-based summarization of multiple documents , 2004 .

[17]  Jie Tang,et al.  Multi-topic Based Query-Oriented Summarization , 2009, SDM.

[18]  Hua Li,et al.  Document Summarization Using Conditional Random Fields , 2007, IJCAI.

[19]  Xiaojun Wan,et al.  Graph-Based MultiModality Learning for Topic-Focused Multi-Document Summarization , 2009 .

[20]  Chin-Yew Lin,et al.  ROUGE: A Package for Automatic Evaluation of Summaries , 2004, ACL 2004.

[21]  Jiebo Luo,et al.  Enhancing semantic and geographic annotation of web images via logistic canonical correlation regression , 2009, ACM Multimedia.

[22]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[23]  G. Leuba,et al.  Changes in volume, surface estimate, three-dimensional shape and total number of neurons of the human primary visual cortex from midgestation until old age , 1994, Anatomy and Embryology.

[24]  H. P. Edmundson,et al.  New Methods in Automatic Extracting , 1969, JACM.

[25]  Vibhu O. Mittal,et al.  Query-Relevant Summarization using FAQs , 2000, ACL.

[26]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[27]  Paul Smolensky,et al.  Information processing in dynamical systems: foundations of harmony theory , 1986 .

[28]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[29]  William John Clancey,et al.  Transfer of rule-based expertise through a tutorial dialogue , 1979 .

[30]  Hoa Trang Dang,et al.  Overview of DUC 2005 , 2005 .

[31]  Ivor W. Tsang,et al.  Using large-scale web data to facilitate textual query based retrieval of consumer photos , 2009, MM '09.

[32]  Hans Peter Luhn,et al.  The Automatic Creation of Literature Abstracts , 1958, IBM J. Res. Dev..

[33]  Alberto Del Bimbo,et al.  Deep networks for audio event classification in soccer videos , 2009, 2009 IEEE International Conference on Multimedia and Expo.

[34]  Ryan T. McDonald.  A Study of Global Inference Algorithms in Multi-document Summarization , 2007, ECIR.

[35]  Jaime Carbonell,et al.  Multi-Document Summarization By Sentence Extraction , 2000 .

[36]  Geoffrey E. Hinton.  A Practical Guide to Training Restricted Boltzmann Machines , 2012, Neural Networks: Tricks of the Trade.

[37]  Yann LeCun,et al.  Convolutional Learning of Spatio-temporal Features , 2010, ECCV.

[38]  Rui Zhang,et al.  Learning object classes from image thumbnails through deep neural networks , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[39]  A. Jefferson Offutt,et al.  An Empirical Evaluation , 1994 .

[40]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[41]  Kam-Fai Wong,et al.  Extractive Summarization Using Supervised and Semi-Supervised Learning , 2008, COLING.

[42]  Xiaojun Wan,et al.  Single Document Keyphrase Extraction Using Neighborhood Knowledge , 2008, AAAI.