Leveraging Pyramids

The Pyramid system associates importance ratings with individual sentences in evaluated summaries. Given that many summarization systems compose their output from source document sentences with limited or no editing, it is often possible to propagate Pyramid ratings back to the source document and tag source sentences with one measure of their suitability to a generic summary. In the case of Pyramid evaluations of summaries of twenty topics in DUC 2005, this could be done in sufficiently many instances to make the effort worthwhile: 14% of nineteen thousand source document sentences were annotated with importance ratings.