Building better corpora for summarisation

Annotated corpora have proved essential in many areas of NLP, and over the years have been successfully exploited for a wide range of tasks in Computational Linguistics, including part-of-speech tagging, parsing and information extraction. One field in which they have been particularly useful is automatic summarisation. Within this field, annotated corpora are mainly used for machine learning, to learn patterns for the extraction of important (and other) information from texts, as well as for the more complex task of evaluation of summarisation methods (Edmundson, 1969; Kupiec, Pederson and Chen, 1995; Zechner, 1996; Marcu, 1997). When annotating corpora, one (accurate) method is to employ humans to indicate those parts of text to be annotated with whatever information necessary. These human-selected units of text can then be used as a gold standard by which to measure the performance of a system, as well as for discerning which types of units are chosen or discarded by humans during the summarisation process. There are semi-automatic (Orasan, 2002) and automatic (Jing and McKeown, 1999; Marcu, 1999) ways to annotate corpora, but given that we are investigating new types of information to be marked, manual annotation is most appropriate here. Despite the fact that they are vital to the field, corpora annotated for summarisation are relatively sparse, and those resources which do exist do not contain as much information as they could.