A Modeling Method for Video Stream Based on Sequence Structure Graph

Recent work in video content analysis focuses on detecting shot boundaries. In the parsing process, video streams are segmented into shots, and each shot may be represented visually by one or more key frames extracted from it. These key frames can be used for video browsing purpose. But many video programs have underlying story structures and key frame representation cannot provide a story line hiding in the raw video stream. In this paper we propose techniques to build the model of video stream and analyze relations between shots only using low-level visual features. First, a video model is constructed to show the story line of video. The model is based on graph model and it can show the switching order of cameras. Then, a clustering algorithm is designed to mark video decomposition units that are recorded by the same camera. Here, both similarity and continuity of video shots are detected and scored, and all video shots that come from same camera can be merged into same video class. In addition, a constant C, which limits most switching time between two shots of same video class, is applied to improve clustering result and minimize time consuming. Lastly, a sequence structure graph that can describe story line is defined. The sequence structure graph of video stream directly shows the story development clue-story line. Results show that the sequence structure graph offers better organization of video that cannot be achieved with existing shot boundary detection schemes. In addition, the graph based analysis scheme offers a compact representation of video content and can be used for the efficient no-linear accessing of video streams.