Rushes summarization based on color, motion and face

In this paper, we present a method for the Rushes Summarization task which is one of tasks of TRECVID 2008. In the proposed method, first an input video is decomposed into shots by comparing consecutive frames. Then, these shots are grouped by the k-means method, using color, motion and faces as features. In the preliminary experiments, we compared three systems which employed the following feature combinations: "color", "color and motion" and "color, motion and faces". As a result, we found out that motion features and face features were effective. Our results of Rushes Summarization 2008 were a little below the median regarding IN (inclusion ratio of ground truth) and JU (lack of junk shots), but were above the median regarding TE (pleasant tempo). Then, to improve IN and JU, we modified the method to detect clapper boards by introducing visual feature in addition to sound feature. The additional experiment regarding the modification after submission shows that it improved the results.