Automatic video temporal segmentation based on multiple features

This paper investigates automatic video temporal segmentation techniques, also named shot boundary detection (SBD) techniques. Firstly, the existing SBD algorithms are reviewed in detail. Then, a new SBD algorithm is proposed aiming to obtain fast and accurate detection, and its performances are evaluated and compared with existing works. This algorithm computes the frame difference/similarity by such simple features as pixel difference and histogram difference, adopts motion-based difference to resist camera or object movements in the same shot and uses the flash detection to avoid false positives caused by light changes or flashes. The adopted features are computational efficient, and the combination of various features improve the detection accuracy. These properties make the algorithm suitable for real-time applications, such as broadcasted news segmentation.

[1]  Ramesh C. Jain,et al.  Knowledge-guided parsing in video databases , 1993, Electronic Imaging.

[2]  Nobuyuki Yagi,et al.  Shot Boundary Detection at TRECVID 2007 , 2007, TRECVID.

[3]  Behzad Shahraray,et al.  Scene change detection and content-based sampling of video sequences , 1995, Electronic Imaging.

[4]  Akio Nagasaka,et al.  Automatic Video Indexing and Full-Video Search for Object Appearances , 1991, VDB.

[5]  Bede Liu,et al.  Temporal segmentation of video using frame and histogram space , 2000, IEEE Transactions on Multimedia.

[6]  Atreyi Kankanhalli,et al.  Automatic partitioning of full-motion video , 1993, Multimedia Systems.

[7]  Narendra Ahuja,et al.  Robust video shot change detection , 1998, 1998 IEEE Second Workshop on Multimedia Signal Processing (Cat. No.98EX175).

[8]  Ajay Divakaran.  Multimedia Content Analysis , 2009 .

[9]  Ajay Divakaran.  Multimedia Content Analysis: Theory and Applications , 2008 .

[10]  Bo Zhang,et al.  A Formal Study of Shot Boundary Detection , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[11]  William J. Christmas,et al.  Video Shot Cut Detection using Adaptive Thresholding , 2000, BMVC.

[12]  Keiichiro Hoashi,et al.  SVM-Based Shot Boundary Detection with a Novel Feature , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[13]  Ioannis Pitas,et al.  Information theory-based shot cut/fade detection and video summarization , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  John S. Boreczky,et al.  Comparison of video shot boundary detection techniques , 1996, Electronic Imaging.

[15]  Ba Tu Truong,et al.  New enhancements to cut, fade, and dissolve detection processes in video segmentation , 2000, ACM Multimedia.

[16]  Xinbo Gao,et al.  Unsupervised video-shot segmentation and model-free anchorperson detection for news video story parsing , 2002, IEEE Trans. Circuits Syst. Video Technol..

[17]  Zhu Liu,et al.  Multimedia content analysis-using both audio and visual clues , 2000, IEEE Signal Process. Mag..

[18]  Yong Fang,et al.  A New General Framework for Shot Boundary Detection Based on SVM , 2005, 2005 International Conference on Neural Networks and Brain.

[19]  Thomas D. C. Little,et al.  A digital on-demand video service supporting content-based queries , 1993, MULTIMEDIA '93.

[20]  Ramin Zabih,et al.  A feature-based algorithm for detecting and classifying scene breaks , 1995, MULTIMEDIA '95.

[21]  Jeho Nam,et al.  Detection of gradual transitions in video sequences using B-spline interpolation , 2005, IEEE Transactions on Multimedia.

[22]  A. Murat Tekalp,et al.  Automatic Soccer Video Analysis and Summarization , 2003, IS&T/SPIE Electronic Imaging.

[23]  Ramin Zabih,et al.  A feature-based algorithm for detecting and classifying production effects , 1999, Multimedia Systems.

[24]  Yihong Gong,et al.  Machine Learning for Multimedia Content Analysis (Multimedia Systems and Applications) , 2007 .

[25]  Ramesh C. Jain,et al.  Digital video segmentation , 1994, MULTIMEDIA '94.

[26]  Chong-Wah Ngo,et al.  A robust dissolve detector by support vector machine , 2003, ACM Multimedia.

[27]  Takafumi Miyatake,et al.  IMPACT: an interactive natural-motion-picture dedicated multimedia authoring system , 1991, CHI.

[28]  Arding Hsu,et al.  Image processing on encoded video sequences , 1994, Multimedia Systems.

[29]  Bo Han,et al.  Enhanced Sports Video Shot Boundary Detection Based on Middle Level Features and a Unified Model , 2007, IEEE Transactions on Consumer Electronics.

[30]  Ramesh C. Jain,et al.  Dynamic vision , 1988, [1988 Proceedings] 9th International Conference on Pattern Recognition.