Automatic Multilevel Temporal Video Structuring

In this paper we propose a novel and complete video scene segmentation framework, developed on different structural levels of analysis. Firstly, a shot boundary detection algorithm is introduced that extends the graph partition method with a nonlinear scale space filtering technique which increase the detection efficiency with gains of 7,4% to 9,8% in terms of both precision and recall rates. Secondly, static storyboards are formed based on a leap key frame extraction method that selects a variable number of key frames, adapted to the visual content variation, for each detected shot. Finally using the extracted key frames, spatio-temporal coherent shots are clustered into the same scene based on temporal constraints and with the help of a new concept of neutralized shots. Video scenes are obtained with average precision and recall rates of 86%.

[1]  Majid Mirmehdi,et al.  Video cut detection using frequency domain correlation , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[2]  Xiang Fu,et al.  An Improved Histogram Based Image Sequence Retrieval Method , 2009 .

[3]  Wolfgang Effelsberg,et al.  Video abstracting , 1997, CACM.

[4]  Bo Zhang,et al.  A Formal Study of Shot Boundary Detection , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[5]  Yaser Sheikh,et al.  On the use of computable features for film classification , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[6]  Bernard Merialdo,et al.  VERT : A METHOD FOR AUTOMATIC EVALUATION OF VIDEO SUMMARIES , 2010 .

[7]  John R. Kender,et al.  Video Summaries through Mosaic-Based Shot and Scene Clustering , 2002, ECCV.

[8]  Andreas Girgensohn,et al.  Time-Constrained Keyframe Selection Technique , 2004, Multimedia Tools and Applications.

[9]  Ullas Gargi,et al.  Performance characterization of video-shot-change detection methods , 2000, IEEE Trans. Circuits Syst. Video Technol..

[10]  Argyris Kalogeratos,et al.  Movie segmentation into scenes and chapters using locally weighted bag of visual words , 2009, CIVR '09.

[11]  Bernard Mérialdo,et al.  VERT: automatic evaluation of video summaries , 2010, ACM Multimedia.

[12]  Stephen W. Smoliar,et al.  An integrated system for content-based video retrieval and browsing , 1997, Pattern Recognit..

[13]  Ramin Zabih,et al.  A feature-based algorithm for detecting and classifying scene breaks , 1995, MULTIMEDIA '95.

[14]  Nikolas P. Galatsanos,et al.  Video rushes summarization using spectral clustering and sequence alignment , 2008, TVS '08.

[15]  Alan Hanjalic,et al.  An integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis , 1999, IEEE Trans. Circuits Syst. Video Technol..

[16]  Chong-Wah Ngo,et al.  Motion-Based Video Representation for Scene Change Detection , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[17]  Atreyi Kankanhalli,et al.  Automatic partitioning of full-motion video , 1993, Multimedia Systems.

[18]  Yuncai Liu,et al.  Video scene segmentation and semantic representation using a novel scheme , 2009, Multimedia Tools and Applications.

[19]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[20]  David Bull,et al.  Scene change detection algorithms for content-based video indexing and retrieval , 2001 .

[21]  Tamara G. Kolda,et al.  Graph partitioning models for parallel computing , 2000, Parallel Comput..

[22]  Yasuo Ariki,et al.  Highlight scene extraction in real time from baseball live video , 2003, MIR '03.

[23]  Andreas Girgensohn,et al.  Time-Constrained Keyframe Selection Technique , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[24]  Alan Hanjalic,et al.  Automated high-level movie segmentation for advanced video-retrieval systems , 1999, IEEE Trans. Circuits Syst. Video Technol..