Bayesian Summarization at DUC and a Suggestion for Extrinsic Evaluation

We describe our entry into the Document Understanding Conference competition for evaluating query-focused multidocument summarization systems. Our system is based on a Bayesian QueryFocused Summarization model, similar to the system we entered into the MSE competition. This paper begins by describing the (few) differences between our DUC system and our MSE system and describes our placement in the competition. The remainder of this paper argues in favor of performing extrinsic evaluation of summarization systems, and suggests a method for doing so.