论文信息 - Agreement in Human Factoid Annotation for Summarization Evaluation

Agreement in Human Factoid Annotation for Summarization Evaluation

Factoid analysis was introduced by (van Halteren and Teufel, 2003) as an objective, yet semantics-oriented way of measuring overlap of information rather than surface strings in summaries. In this paper, we report on annotation experiments with two sets of summaries, and on a factoid-pairing program which finds correlations between factoids semi-automatically.

Hans van Halteren | Simone Teufel

[1] Eduard Hovy,et al. Manual and automatic evaluation of summaries , 2002, ACL 2002.

[2] Mark T. Maybury,et al. Advances in Automatic Text Summarization , 1999 .

[3] Wai Lam,et al. Developing Infrastructure for the Evaluation of Single and Multi-document Summarization Systems in a Cross-lingual Environment , 2002, LREC.

[4] Kathleen R. McKeown,et al. Summarization Evaluation Methods: Experiments and Analysis , 1998 .

[5] Simone Teufel,et al. Examining the consensus between human summaries: initial experiments with factoid analysis , 2003, HLT-NAACL 2003.

[6] Karen Spärck Jones. Automatic summarising: factors and directions , 1998, ArXiv.

[7] Inderjeet Mani,et al. The Tipster Summac Text Summarization Evaluation , 1999, EACL.

[8] Klaus Zechner,et al. Fast Generation of Abstracts from General Domain Text Corpora by Extracting Relevant Sentences , 1996, COLING.

[9] Gustave J. Rath,et al. The formation of abstracts by the selection of sentences , 1961 .