Video scene decomposition with the motion picture parser

A motion picture can be modeled as a composition of many scenes where each scene is comprised of multiple shots. Thus, a conventional movie is a sequential aggregation of a large number of disparate image sequences. Within each image sequence or shot, there is consistency in image content and dynamics. This consistency in dynamics can be used in identifying scene changes for video segment decomposition and for techniques to improve data compression. We have developed an algorithm to use these dynamics for scene change detection and the decomposition of video streams into constituent logical shots. The algorithm uses intraframe image complexity and identifies scene transitions by considering short-term temporal dynamics. The algorithm has shown to be effective for detecting both abrupt scene changes (cuts) as well as smooth scene changes (fades and dissolves). This algorithm is used in an application we have developed called the Motion Picture Parser (MPP). The MPP automates the process of tagging segments of motion-JPEG-compressed movies. Segments are also tagged for subsequent semantic content-based retrieval in units of shots and scenes. The MPP application consists of a graphical user interface with various editing controls.