Video summarization by redundancy removing and content ranking

In order to help the user to grasp the long video content quickly, this paper proposes a novel video summarization approach based on redundancy removal and content ranking. By video parsing and cast indexing, the approach first constructs a story board to let user know about the main scenes and the main actors in the video. Then it generates a "story-constraint summary" by key frame clustering and repetitive segment detection. To shorten the video summary length to a target length, our approach constructs a "time-constraint summary" by important factor based content ranking. Extensive experiments are carried out on TV series, movies, and cartoons. Good results demonstrate the effectiveness of the proposed method.

[1]  Yue Gao,et al.  THU-ICRC at rush summarization of TRECVID 2007 , 2007, TVS '07.

[2]  Mubarak Shah,et al.  Detection and representation of scenes in videos , 2005, IEEE Transactions on Multimedia.

[3]  Jun Wu,et al.  Tsinghua University at TRECVID 2004: Shot Boundary Detection and High-Level Feature Extraction , 2004, TRECVID.

[4]  Tao Wang,et al.  Cast indexing for videos by NCuts and page ranking , 2007, CIVR '07.

[5]  Werner Bailer,et al.  Skimming rushes video using retake detection , 2007, TVS '07.

[6]  M S Waterman,et al.  Identification of common molecular subsequences. , 1981, Journal of molecular biology.

[7]  Chong-Wah Ngo,et al.  Rushes video summarization by object and event understanding , 2007, TVS '07.

[8]  Wei-Hao Lin,et al.  Clever clustering vs. simple speed-up for summarizing rushes , 2007, TVS '07.

[9]  Yuan Li,et al.  Robust Head Tracking with Particles Based on Multiple Cues Fusion , 2006, ECCV Workshop on HCI.

[10]  Paul Over,et al.  The trecvid 2008 BBC rushes summarization evaluation , 2008, TVS '08.

[11]  Mubarak Shah,et al.  Scene detection in Hollywood movies and TV shows , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[12]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[13]  Shingo Uchihashi,et al.  Video Manga: generating semantically meaningful video summaries , 1999, MULTIMEDIA '99.

[14]  Eduardo Romero,et al.  Biomedical Image Analysis and Machine Learning Technologies: Applications and Techniques , 2009 .

[15]  Tao Wang,et al.  Caption-aided speech detection in videos , 2008, 2008 IEEE International Conference on Acoustics, Speech and Signal Processing.

[16]  Aggelos K. Katsaggelos,et al.  Rate-distortion optimal video summary generation , 2005, IEEE Transactions on Image Processing.

[17]  Masaharu Ogawa,et al.  A highlight scene detection and video summarization system using audio feature for a personal video recorder , 2005, IEEE Transactions on Consumer Electronics.

[18]  Anindya Sarkar,et al.  Feature fusion and redundancy pruning for rush video summarization , 2007, TVS '07.

[19]  Peng Wang,et al.  Scene Segmentation and Categorization Using NCuts , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Xin Liu,et al.  Video summarization with minimal visual content redundancies , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[21]  Nobuyuki Yagi,et al.  Estimation of camera parameters from image sequence for model-based video coding , 1994, IEEE Trans. Circuits Syst. Video Technol..

[22]  Paul Over,et al.  The trecvid 2007 BBC rushes summarization evaluation pilot , 2007, TVS '07.