Visual Content-Based Segmentation of Talk and Game Shows

Abstract In this article we present a method to remove commercials from talk and game show videos and to segment these videos into host and guest shots. In our approach, we rely mainly on information contained in shot transitions, rather than analyzing the scene content of individual frames. We utilize the inherent difference in scene structure of commercials and talk shows to differentiate between them. Similarly, we make use of the well-defined structure of talk shows, which can be exploited to classify shots as host or guest shots. The entire show is first segmented into camera shots based on colour histogram. Then we construct a data-structure (shot connectivity graph) that links similar shots over time. Analysis of the shot connectivity graph helps us to automatically separate commercials from program segments. This is done by first detecting stories and then assigning a weight to each story based on its likelihood of being a commercial. Further analysis on stories is done to distinguish shots of the hosts from shots of the guests. We have tested our approach on several full-length shows (including commercials) and have achieved video segmentation with high accuracy. The whole scheme is fast and works even on low- quality video (160x120 pixel images at 5 Hz).