Toward Extractive Summarization of Multimodal Documents

Summarization research has focused on text, and relatively little attention has been given to the summarization of multimodal documents. If extractive summarization techniques are to be used on multimodal documents containing information graphics (bar charts, line graphs, etc.), then a strategy must be devised both for extracting the high-level content of the information graphics and for identifying where that content is relevant in the article’s text. This paper gives an overview of our prior work on constructing a summary of an information graphic and presents our new research on methods for selecting paragraphs in a multimodal document that are most relevant to a constituent information graphic. The results demonstrate that our methods are far superior to possible baseline methods and that our work advances the use of extractive techniques for summarizing multimodal documents.