Shot boundary detection based on block-wise principal component analysis

Abstract. With the rapid development of digital video, shot boundary detection (SBD) has attracted much attention since it is the fundamental preprocessing for video indexing, annotation, retrieval, and other content-based operations. However, most state-of-the-art SBD methods are based on the spatial features of video image, and the overall characteristics of video shots are not fully considered. We propose a feature extraction method based on shot characteristics and a more robust SBD process. First, a video is divided into several segments, the segments containing consecutive video frames inside a shot are considered as training segments and others are called candidate segments. Afterward, using block-wise principal component analysis on the training segments, shot eigenspaces are established. The video frames in the candidate segment are then projected onto the corresponding shot eigenspace to extract the feature vectors. Finally, analysis and pattern matching for feature vectors are performed to extract the video shot boundary. Experiments on TRECVID test data demonstrate that the mean values of F1 in cut transition detection and gradual transition (GT) detection of our method are 0.901 and 0.866, respectively, obviously higher than the values of the compared methods, especially in GT detection, thus providing better accuracy in SBD.