Searching Movies Based on User Defined Semantic Events

The number, and size, of digital video databases is continuously growing. Unfortunately, most, if not all, of the video content in these databases is stored without any sort of indexing or analysis and without any associated metadata. If any of the videos do have metadata, then it is usually the result of some manual annotation process rather than any automatic indexing. Locating clips and browsing content is difficult, time consuming and generally inefficient. The task of managing a set of movies is particularly difficult given their innovative creation process and the individual style of directors. This paper proposes a method of searching video data in order to retrieve semantic events thereby facilitating management of video databases. An interface is created which allows users to perform searching using the proposed method. In order to assess the searching method, this interface is used to conduct a set of experiments in which users are timed completing a set of tasks using both the searching method and an alternate, keyframe based, retrieval method. These experiments evaluate the searching method, and demonstrate it’s versatility.

[1]  Boon-Lock Yeo,et al.  Time-constrained clustering for segmentation of video into story units , 1996, Proceedings of 13th International Conference on Pattern Recognition.

[2]  Shih-Fu Chang,et al.  Determining computable scenes in films and their structures using audio-visual memory models , 2000, ACM Multimedia.

[3]  Mubarak Shah,et al.  Scene detection in Hollywood movies and TV shows , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[4]  Lei Chen,et al.  Incorporating Audio Cues into Dialog and Action Scene Extraction , 2003, IS&T/SPIE Electronic Imaging.

[5]  Wallapak Tavanapong,et al.  ShotWeave: A Shot Clustering Technique for Story Browsing for Large Video Databases , 2002, EDBT Workshops.

[6]  Thomas S. Huang,et al.  Constructing table-of-content for videos , 1999, Multimedia Systems.

[7]  Boon-Lock Yeo,et al.  Video visualization for compact presentation and fast browsing of pictorial content , 1997, IEEE Trans. Circuits Syst. Video Technol..

[8]  Hang-Bong Kang.  Emotional event detection using relevance feedback , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[9]  Ying Li,et al.  Movie Event Detection by Using Audio Visual Information , 2001, IEEE Pacific Rim Conference on Multimedia.

[10]  Mubarak Shah,et al.  A Framework for Semantic Classification of Scenes Using Finite State Machines , 2004, CIVR.

[11]  Rainer Lienhart,et al.  Scene Determination Based on Video and Audio Features , 2004, Multimedia Tools and Applications.

[12]  Noel E. O'Connor,et al.  Dialogue Sequence Detection in Movies , 2005, CIVR.

[13]  Jeho Nam,et al.  Audio-visual content-based violent scene characterization , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[14]  Noel E. O'Connor,et al.  Dialogue scene detection in movies using low and mid-level visual features , 2004 .

[15]  Noel E. O'Connor,et al.  Action Sequence Detection in Motion Pictures , 2004, EWIMT.