Multi-Document Summarization Reflecting Information Needs on Subjectivity

In this paper, we present our experiments on improving multi-document summarization by reflecting information needs on subjectivity. Subjectivity is an essential aspect for better understanding of information needs. Our approach is based on sentence extraction, weighted by sentence type annotation, and combined with polarity term frequencies. From the DUC 2005 dataset, which focused on summarization for English documents, we selected 10 topics expressing information needs for subjective information and evaluated our results with two types of evaluation metrics: ROUGE (Recall-Oriented Understudy for Gisting Evaluation) and BE (Basic Elements). For 10 topics, we found improvements of 10.2 % in the ROUGE-2 score, when compared to the baseline system with no analysis of topics. With failure analysis, we found the topics with improvements of ROUGE and BE scores contained effective subjective keywords.

[1]  V. Hatzivassiloglou.  Lists of manually and automatically identified gradable, polar, and dynamic adjectives, gzipped tar file , 2000 .

[2]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[3]  Janyce Wiebe,et al.  Effects of Adjective Orientation and Gradability on Sentence Subjectivity , 2000, COLING.

[4]  Eduard Hovy,et al.  Evaluating DUC 2005 using Basic Elements , 2005 .

[5]  N. Kando,et al.  Analysis of Multi-Document Viewpoint Summarization Using Multi-Dimensional Genres , 2004 .

[6]  Noriko Kando,et al.  User-Focused Multi-Document Summarization with Paragraph Clustering and Sentence-Type Filtering , 2004, NTCIR.

[7]  Janyce Wiebe,et al.  Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis , 2005, HLT.

[8]  Manabu Okumura,et al.  Text Summarization Challenge 2 text summarization evaluation at NTCIR workshop 3 , 2004, SIGF.

[9]  J. R. Quinlan,et al.  Data Mining Tools See5 and C5.0 , 2004 .

[10]  Noriko Kando,et al.  Multi-Document Viewpoint Summarization Focused on Facts, Opinion and Knowledge , 2006, Computing Attitude and Affect in Text.

[11]  Vasileios Hatzivassiloglou,et al.  Predicting the Semantic Orientation of Adjectives , 1997, ACL.

[12]  Hong Yu,et al.  Towards Answering Opinion Questions: Separating Facts from Opinions and Identifying the Polarity of Opinion Sentences , 2003, EMNLP.

[13]  Thorsten Joachims,et al.  Learning to classify text using support vector machines - methods, theory and algorithms , 2002, The Kluwer international series in engineering and computer science.

[14]  Hoa Trang Dang,et al.  Overview of DUC 2005 , 2005 .

[15]  Ellen Riloff,et al.  Learning Extraction Patterns for Subjective Expressions , 2003, EMNLP.

[16]  Noriko Kando,et al.  Multi-Document Summarization with Subjectivity Analysis at DUC 2005 , 2005 .

[17]  Manabu Okumura,et al.  Text summarization challenge 2: text summarization evaluation at NTCIR workshop 3 , 2001, HLT-NAACL 2003.

[18]  Kathleen R. McKeown,et al.  Applying the Pyramid Method in DUC 2005 , 2005 .

[19]  Janyce Wiebe,et al.  Learning Subjective Language , 2004, CL.