Dynamic video summarization using two-level redundancy detection

The mushroom growth of video information, consequently, necessitates the progress of content-based video analysis techniques. Video summarization, aiming to provide a short video summary of the original video document, has drawn much attention these years. In this paper, we propose an algorithm for video summarization with a two-level redundancy detection procedure. By video segmentation and cast indexing, the algorithm first constructs story boards to let users know main scenes and cast (when this is a video with cast) in the video. Then it removes redundant video content using hierarchical agglomerative clustering in the key frame level. The impact factors of scenes and key frames are defined, and parts of key frames are selected to generate the initial video summary. Finally, a repetitive frame segment detection procedure is designed to remove redundant information in the initial video summary. Results of experimental applications on TV series, movies and cartoons are given to illustrate the proposed algorithm.

[1]  De Xu,et al.  An approach to generating two-level video abstraction , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[2]  Raimondo Schettini,et al.  Supervised and unsupervised classification post-processing for visual video summaries , 2006, IEEE Transactions on Consumer Electronics.

[3]  Kin-Man Lam,et al.  A new key frame representation for video segment retrieval , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[4]  Wei Xiong,et al.  Query by video clip , 1998, Proceedings. Fourteenth International Conference on Pattern Recognition (Cat. No.98EX170).

[5]  Irena Koprinska,et al.  Temporal video segmentation: A survey , 2001, Signal Process. Image Commun..

[6]  A. Murat Tekalp,et al.  Two-stage hierarchical video summary extraction to match low-level user browsing preferences , 2003, IEEE Trans. Multim..

[7]  Tiecheng Liu,et al.  Content-Adaptive Video Summarization Combining Queueing and Clustering , 2006, 2006 International Conference on Image Processing.

[8]  M S Waterman,et al.  Identification of common molecular subsequences. , 1981, Journal of molecular biology.

[9]  Amit K. Roy-Chowdhury,et al.  Summarization and Indexing of Human Activity Sequences , 2006, 2006 International Conference on Image Processing.

[10]  Yuan Li,et al.  Robust Head Tracking with Particles Based on Multiple Cues Fusion , 2006, ECCV Workshop on HCI.

[11]  Peng Wang,et al.  Scene Segmentation and Categorization Using NCuts , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Ioannis Pitas,et al.  Information theory-based shot cut/fade detection and video summarization , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[13]  Xin Liu,et al.  Video summarization with minimal visual content redundancies , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[14]  S. Shipman,et al.  Architecture for video summarization services over home networks and the Internet , 2006, 2006 Digest of Technical Papers International Conference on Consumer Electronics.

[15]  Guizhong Liu,et al.  A Multiple Visual Models Based Perceptive Analysis Framework for Multilevel Video Summarization , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[16]  Xindong Wu,et al.  Sequential association mining for video summarization , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[17]  Mubarak Shah,et al.  Detection and representation of scenes in videos , 2005, IEEE Transactions on Multimedia.

[18]  Tao Wang,et al.  Cast indexing for videos by NCuts and page ranking , 2007, CIVR '07.

[19]  Chong-Wah Ngo,et al.  Automatic video summarization by graph modeling , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[20]  Jacob Scharcanski,et al.  Hierarchical Summarization of Diagnostic Hysteroscopy Videos , 2006, 2006 International Conference on Image Processing.

[21]  Shingo Uchihashi,et al.  Video Manga: generating semantically meaningful video summaries , 1999, MULTIMEDIA '99.

[22]  Majid Mirmehdi,et al.  A shortest path representation for video summarisation , 2003, 12th International Conference on Image Analysis and Processing, 2003.Proceedings..

[23]  Michael R. Lyu,et al.  Video summarization by spatial-temporal graph optimization , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[24]  Sang Hyun Kim,et al.  An efficient algorithm for video sequence matching using the modified Hausdorff distance and the directed divergence , 2002, IEEE Trans. Circuits Syst. Video Technol..

[25]  Aggelos K. Katsaggelos,et al.  Rate-distortion optimal video summary generation , 2005, IEEE Transactions on Image Processing.

[26]  Riccardo Leonardi,et al.  Extraction of Significant Video Summaries by Dendrogram Analysis , 2006, 2006 International Conference on Image Processing.

[27]  Masaharu Ogawa,et al.  A highlight scene detection and video summarization system using audio feature for a personal video recorder , 2005, IEEE Transactions on Consumer Electronics.

[28]  Jung-Hwan Oh,et al.  Scenario based dynamic video abstractions using graph matching , 2005, MULTIMEDIA '05.

[29]  Janko Calic,et al.  Efficient Layout of Comic-Like Video Summaries , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Yuxin Peng,et al.  Clip-based similarity measure for query-dependent clip retrieval and video summarization , 2006, IEEE Trans. Circuits Syst. Video Technol..

[31]  Ajay Divakaran,et al.  Broadcast Video Program Summarization using Face Tracks , 2006, 2006 IEEE International Conference on Multimedia and Expo.

[32]  Mubarak Shah,et al.  Scene detection in Hollywood movies and TV shows , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[33]  Ramin Zabih,et al.  A feature-based algorithm for detecting and classifying production effects , 1999, Multimedia Systems.