With the increasing use and availability of digital video, shot boundary detection, as a fundamental step in automatic video content analysis and retrieval, has received extensive attention in recent years. But the problem of detecting gradual transitions such as fades, especially in the compressed domain, has yet to be solved. In this paper, a new robust compressed domain fade detection method is proposed, which is based on the luminance weighting factors available at slice level in H.264 compressed video. Simplicity and working directly in the H.264 compressed domain with high precision and recall are the main advantages of the novel algorithm.

We present a general approach to temporal media segmentation using supervised classification. Given standard low-level features representing each time sample, we build intermediate features via pairwise similarity. The intermediate features comprehensively characterize local temporal structure, and are input to an efficient supervised classifier to identify shot boundaries. We integrate discriminative feature selection based on mutual information to enhance performance and reduce processing requirements. Experimental results using large-scale test sets provided by the TRECVID evaluations for abrupt and gradual shot boundary detection are presented, demonstrating excellent performance

Videos are composed of many shots that are caused by different camera operations, e.g., on/off operations and switching between cameras. One important goal in video analysis is to group the shots into temporal scenes, such that all the shots in a single scene are related to the same subject, which could be a particular physical setting, an ongoing action or a theme. In this paper, we present a general framework for temporal scene segmentation in various video domains. The proposed method is formulated in a statistical fashion and uses the Markov chain Monte Carlo (MCMC) technique to determine the boundaries between video scenes. In this approach, a set of arbitrary scene boundaries are initialized at random locations and are automatically updated using two types of updates: diffusion and jumps. Diffusion is the process of updating the boundaries between adjacent scenes. Jumps consist of two reversible operations: the merging of two scenes and the splitting of an existing scene. The posterior probability of the target distribution of the number of scenes and their corresponding boundary locations is computed based on the model priors and the data likelihood. The updates of the model parameters are controlled by the hypothesis ratio test in the MCMC process, and the samples are collected to generate the final scene boundaries. The major advantage of the proposed framework is two-fold: 1) it is able to find the weak boundaries as well as the strong boundaries, i.e., it does not rely on the fixed threshold; 2) it can be applied to different video domains. We have tested the proposed method on two video domains: home videos and feature films, and accurate results have been obtained

Videos are composed of many shots that are caused by different camera operations, e.g., on/off operations and switching between cameras. One important goal in video analysis is to group the shots into temporal scenes, such that all the shots in a single scene are related to the same subject, which could be a particular physical setting, an ongoing action or a theme. In this paper, we present a general framework for temporal scene segmentation in various video domains. The proposed method is formulated in a statistical fashion and uses the Markov chain Monte Carlo (MCMC) technique to determine the boundaries between video scenes. In this approach, a set of arbitrary scene boundaries are initialized at random locations and are automatically updated using two types of updates: diffusion and jumps. Diffusion is the process of updating the boundaries between adjacent scenes. Jumps consist of two reversible operations: the merging of two scenes and the splitting of an existing scene. The posterior prob- ability of the target distribution of the number of scenes and their corresponding boundary locations is computed based on the model priors and the data likelihood. The updates of the model parameters are controlled by the hypothesis ratio test in the MCMC process, and the samples are collected to generate the final scene boundaries. The major advantage of the proposed framework is two-fold: 1) it is able to find the weak boundaries as well as the strong boundaries, i.e., it does not rely on the fixed threshold; 2) it can be applied to different video domains. We have tested the proposed method on two video domains: home videos and feature films, and accurate results have been obtained. Index Terms—Markov chain Monte Carlo, video scene segmen- tation.

Videos are composed of many shots that are caused by different camera operations, e.g., on/off operations and switch- ing between cameras. One important goal in video analysis is to group the shots into temporal scenes, such that all the shots in a single scene are related to the same subject, which could be a particular physical setting, an on-going action or a theme. In this paper, we present a general framework for temporal scene segmentation in various video domains. The proposed method is formulated in a statistical fashion and uses the Markov chain Monte Carlo (MCMC) technique to determine the boundaries between video scenes. In this approach, a set of arbitrary scene boundaries are initialized at random locations and are automat- ically updated using two types of updates: diffusion and jumps. Diffusion is the process of updating the boundaries between adjacent scenes. Jumps consist of two reversible operations: the merging of two scenes and the splitting of an existing scene. The posterior probability of the target distribution of the number of scenes and their corresponding boundary locations is computed based on the model priors and the data likelihood. The updates of the model parameters are controlled by the hypothesis ratio test in the MCMC process, and the samples are collected to generate the final scene boundaries. The major advantage of the proposed framework is two-fold: (1) it is able to find the weak boundaries as well as the strong boundaries, i.e., it does not rely on the fixed threshold; (2) it can be applied to different video domains. We have tested the proposed method on two video domains: Home Videos and Feature Films, and accurate results have been obtained. provides the chapters that correspond to the different sub- themes of the movies. In television videos, segmentation can be used to separate the commercials from the regular programs. In news broadcast programs, segmentation can be used to identify different news stories. In home videos, scene segmentation may help the consumers to logically organize the videos related to the different events (e.g., birthday, graduation, weddings, or vacation (e.g. city tours, sightseeing)).

Various algorithms have been proposed in last decade, for temporal segmentation of digital videos. Comparison of the work of different research groups working in this area can be seen from TRECVID 2005. It is clear from comparison that no method is suitable for all types of video to detect all types of gradual transitions. Further many methods are either computation intensive or too much false detection reduces their efficiency. In this work, a method based on region matching using area correlation is proposed. This method can be used in prior to any other method to reduce the false detections. In this method, different frames of video are divided into different regions based on their statistical similarity. For each region in two consecutive frames, area correlation function is calculated. This area correlation energy function is used to detect the similarity among the regions. Weighted sum of correlation function in all the mismatched regions and location of these are used to classify the edit shots of a video into different types such as hard cuts, fades, wipes etc,. This method is tested on different videos and reasonable performance is achieved. It can also be used as pre-classifier in prior to other methods to reduce their false detections.

This report surveys the state of the art of analysis algorithms and tools for audiovisual content and discusses their feasibility and practical usability, as well as the interdependencies between the various tools.

This paper presents the Shot Boundary Detection system developed by LaBRI in the context of “Rough Indexing” paradigm. We work on compressed streams and we use only I and P frames information, (DC coefficients of I-Frames, motion vectors of PFrames and DC coefficients of prediction error) which allow us to be faster than many equivalent systems (10 times faster than real-time on TRECVID2003 test set, and 3 times faster on 2004, because MPEG files structure is composed of only I and P frames). In this context the application was not developed to classify shot change transition effects, the initial goal was to allow a real-time and intelligent browsing in video content for common users. The detection is performed in two stages: - Robust Global Camera Motion Estimation - Detection of P-Frame peaks (computation of motion and frame statistics), and of I-Frames (measuring similarity on successive compensated I frames). As we work with two types of frames (I and P), we associate two statistical models which give us two sets of ratio and threshold to calibrate the detector. The first TRECVID participation of LaBRI implies an evolution of the application for transitions effects distinction, which induces two new thresholds to calibrate. We generally obtain equivalent values of Recall and Precision (0.72 on TRECVID 2003 test set). On TRECVID 2004 test set we obtain as best runs ri-3: 0.723(Recall) and 0.606(Precision); and ri-4: 0.703(Recall) and 0.635(Precision).

This paper describes an integrated system for the acquisition, automatic annotation and Web publication of television broadcast news programmes named ANTS (automatic newscast transcription system). The system consists of several analysis components integrated in a unified architecture. Users have the possibility to access a large daily-growing database of news stories from the main national channels, all identified, categorised and published in a fully automatic way. The system identifies story boundaries, extracts texts from spoken content, classifies stories by subject and links external relevant information coming from the Web. Performances of the system have been evaluated in a real-life scenario, by providing an instance of the system to a panel of professional users inside RAI. The strength of the approach behind ANTS consists in the ability to integrate several heterogeneous analysis components in a performant and ready-for-production environment, capable of elaborating many hours of material per day without significant service drops and with sufficiently good accuracy for industrial employment in large broadcasting facilities.

This paper describes Hyper Media News (HMNews), a system for the automated aggregation and consumption of information streams from digital television and the Internet. TV newscasts are automatically segmented, annotated and indexed. Such information is then integrated with those available from Internet blogs, newspapers and press agencies. The end result is a set of innovative information services that supplies retrieval, recommendation and browsing of multi-modal news items across different production paradigms, ranging from traditional professional media, e.g. television and press, to new user-centric media platforms such as social networking sites, internet forums and blogs.

This article presents a scene similarity measure for video content segmentation. In the context of the rough indexing paradigm, we extract only partial information from MPEG compressed streams to measure the similarity of video frames through time. The similarity measure of I‐Frames is defined based on motion compensation of DC images and local contrast computation. The method allows a real‐time segmentation of the video content. © 2006 Wiley Periodicals, Inc. Int J Int Syst 21: 765–783, 2006.

The segmentation of news video into story units is an important step towards effective processing and management of large news video archives. In the story segmentation task in TRECVID 2003, a wide variety of techniques were employed by many research groups to segment over 120-hour of news video. The techniques employed range from simple anchor person detector to soisticated machine learning models based on HMM and Maximum Entropy (ME) approaches. The general results indicate that the judicious use of multi-modality features coupled with rigorous machine learning models could produce effective solutions. This paper presents the algorithms and experience learned in TRECVID evaluations. It also points the way towards the development of scalable technology to process large news video corpuses.

The number, and size, of digital video databases is continuously growing. Unfortunately, most, if not all, of the video content in these databases is stored without any sort of indexing or analysis and without any associated metadata. If any of the videos do have metadata, then it is usually the result of some manual annotation process rather than any automatic indexing. Thus, locating clips and browsing content is difficult, time consuming and generally inefficient. The task of automatically indexing movies is particularly difficult given their innovative creation process and the individual style of many film makers. However, there are a number of underlying film grammar conventions that are universally followed, from a Hollywood blockbuster to an underground movie with a limited budget. These conventions dictate many elements of film making such as camera placement and editing. By examining the use of these conventions it is possible to extract information about the events in a movie. This research aims to provide an approach that creates an indexed version of a movie to facilitate ease of browsing and efficient retrieval. In order to achieve this aim, all of the relevant events contained within a movie are detected and classified into a predefined index. The event detection process involves examining the underlying structure of a movie and utilising audiovisual analysis techniques, supported by machine learning algorithms, to extract information based on this structure. The result is an indexed movie that can be presented to users for browsing/retrieval of relevant events, as well as supporting user specified searching. Extensive evaluation of the indexing approach is carried out. This evaluation indicates efficient performance of the event detection and retrieval system, and also highlights the subjective nature of video content.

The multimedia databases and the need to intuitively handle their content, which meets the user’s requirements with the available content based video indexing and retrieval technology, are the main focus of the research in the field of multimedia and computer vision. The researchers mainly focus on the problem of bridging the “semantic gap” between a user’s need for meaningful retrieval and the current technology for computational analysis and description of the media content. It takes into account both the high complexity of the real-world implementation and user’s need for conceptual video retrieval and browsing. In this paper, the initial work done in this area during last 15 years has been categorized in three generations. The key technologies in each generation are reviewed and characterized based on the standard parameters. It is found that in first and second generation all techniques are semantic less techniques, but in third generation, techniques based on semantics have been evolved. But still most of the techniques are in their infancy and require lots of research for their use in daily applications. In last a solution is proposed for a general purpose multimedia mining application which caters to the needs of the different types of domains.

The growing amount of multimedia data available to the average user has reached a critical phase, where methods for indexing, searching, and efficient retrieval are needed to manage the information overload. Many research works related to this field have been conducted within the last few decades and consequently, some video database models have been proposed. Most of the modern video database models make use of hierarchical structures to organize huge amount of videos to support video retrieval efficiently. Even now, among open research issues, video database access control is still an interesting research area with many proposed models. In this paper, we present a hybrid video database model which is a combination of the hierarchical video database model and annotations. In particular, we extend the original hierarchical indexing mechanism to add frames and salient objects at the lowest granularity level in the video tree with the aim to support multi-level access control. Also, we give users more solutions to query for videos based on the video contents using annotations. In addition, we also suggest the original database access control model to fit the characteristics of video data. Our modified model supports both multiple access control policies, meaning that a user may be affected by multiple polices, and multi-level access control, meaning that an authorization may be specified at any video level. Theoretical analyses and experimental results with real datasets are presented that confirm the correctness and efficiency of our approach.

The global diffusion of the Internet has enabled the distribution of informative content through dynamic media such as RSS feeds and video blogs. At the same time, the decreasing cost of electronic devices has increased the perva- sive availability of the same informative content in the form of digital audiovisual data. This article presents a system for the large-scale unsupervised acquisition, segmentation and index- ing of TV newscasts. In particular, it discusses the principles and performance of the parts of the system dedicated to the detection and segmentation of programmes from the acquired stream. In addition to the core technology, we also introduce and discuss a novel method for assessing the results of story boundaries segmentation algorithms, based on a user-validated measurement. Due to the heterogeneity of current news dis- tribution channels, a further innovative aspect of this article is the description of a framework for multimodal information ag- gregation. The core of this framework is a cross-modal cluster- ing process for which a novel, asymmetric similarity measure is provided. The implemented prototype uses online news articles and TV news programmes as information sources, and provides a multimodal service integrating both contributions. Experimental evaluation of the system proves the effectiveness of the method in the studied case.

TV logo recognition is one of the suggested solutions for preventing unauthorized duplication and redistribution. The major problem of the previous logo recognition is that the matching process against the reference logos requires much time. Since millions of minutes of TV programs are produced and broadcasted every day, an efficient algorithm is necessary. In this paper, we propose lightweight logo recognition using an image moment. An image moment represents a geometrical feature by a numerical low dimensional vector and thus matching can be achieved by a simple distance metric. Since an image moment itself is not so robust to noise, we introduce conspicuousness of a transparent logo according to visual properties of its background and surrounding regions. A validation experiment using the actual TV programs achieves much faster processing than the existing method with only a small degradation in accuracy. In the proposed method, computational complexity is independent of the number of the reference logos and the size of the logo region.

Slow-motion replays are content full segments of broadcast soccer videos. In this paper, we propose an efficient method for detection of slow-motion shots produced by high-speed cameras in soccer broadcasts. A rich set of color, motion, and cinematic features are extracted from compressed video by partial decoding of the MPEG-1 bitstream. Then, slow-motion shots are modeled by SVM classifiers for each shot class. A set of six full-match soccer games is used for training and evaluation of the proposed method. Our algorithm presents satisfactory results along with high speed for slow-motion detection in soccer videos.

Shot segmentation and categorization is one of the most important and primary steps in building efficient and well sorted video storage and retrieval system. News video story segmentation and commercial detection surely deserve special attention for creation of a well-categorized system as different types of news are being broadcast over a variety of news channels in television. However, in order to garner more viewers, different news channels use different spatial and temporal structures for broadcasting news. Keeping this approach in mind, we present an unsupervised method for shot detection, and subsequently, commercial and news shots are classified. The proposed method does not depend on different temporal and spatial structures adopted by different news channels, and our method needs significantly less number of manual parameters.

Logical units are semantic video segments above the shot level. Depending on the common semantics within the unit and data domain, different types of logical unit extraction algorithms have been presented in literature. Topic units are typically extracted for documentaries or news broadcasts while scenes are extracted for narrative-driven video such as feature films, sitcoms, or cartoons. Other types of logical units are extracted from home video and sports. Different algorithms in literature used for the extraction of logical units are reviewed in this paper based on the categories unit type, data domain, features used, segmentation method, and thresholds applied. A detailed comparative study is presented for the case of extracting scenes from narrative-driven video. While earlier comparative studies focused on scene segmentation methods only or on complete news-story segmentation algorithms, in this paper various visual features and segmentation methods with their thresholding mechanisms and their combination into complete scene detection algorithms are investigated. The performance of the resulting large set of algorithms is then evaluated on a set of video files including feature films, sitcoms, children's shows, a detective story, and cartoons.