Video shot detection and characterization for video databases

Abstract The organization of video information for video databases requires segmentation of a video into its constituent shots and their subsequent characterization in terms of content and camera work. In this paper, we look at these two steps using compressed video data directly. For shot detection, we suggest a scheme consisting of comparing intensity, row, and column histograms of successive I frames of MPEG video using the chi-square test. For characterization of segmented shots, we address the problem of classifying shot motion into different categories using a set of features derived from motion vectors of P and B frames of MPEG video. The central component of the proposed shot motion characterization scheme is a decision tree classifier built through a process of supervised learning. Experimental results using a variety of videos are presented to demonstrate the effectiveness of performing shot detection and characterization directly on compressed video.