Toward a conceptual framework of key-frame extraction and storyboard display for video summarization

Two key problems in developing a storyboard are (a) the extraction of video key frames and (b) the display of the storyboard. On the basis of our findings from a preliminary study as well as the results of previous studies on the computerized extraction of key frames and human recognition of images and videos, we propose an algorithm for the extraction of key frames and the structural display of a storyboard. In order to evaluate the proposed algorithm, we conducted an experiment, the results of which suggest that participants produce better summaries of the given videos when they view storyboards that are composed of key frames extracted using the proposed algorithmic method. This finding held, regardless of whether the display pattern used was sequential or structural. In contrast, the experimental results suggest that in the case of employing a mechanical method, the use of a structural display pattern yields greater performance in terms of participants' ability to summarize the given videos. Elaborating on our results, we discuss the practical implications of our findings for video summarization and retrieval. © 2010 Wiley Periodicals, Inc.