Within and Between Shot Information Utilisation in Video Key Frame Extraction

In consequence of the popularity of family video recorders and the surge of Web 2.0, increasing amounts of videos have made the management and integration of the information in videos an urgent and important issue in video retrieval. Key frames, as a high-quality summary of videos, play an important role in the areas of video browsing, searching, categorisation, and indexing. An effective set of key frames should include major objects and events of the video sequence, and should contain minimum content redundancies. In this paper, an innovative key frame extraction method is proposed to select representative key frames for a video. By analysing the differences between frames and utilising the clustering technique, a set of key frame candidates (KFCs) is first selected at the shot level, and then the information within a video shot and between video shots is used to filter the candidate set to generate the final set of key frames. Experimental results on the TRECVID 2007 video dataset have demonstrated the effectiveness of our proposed key frame extraction method in terms of the percentage of the extracted key frames and the retrieval precision.

[1]  Stefanos D. Kollias,et al.  Video content representation using optimal extraction of frames and scenes , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[2]  Zhi-Hua Zhou,et al.  Multi-View Video Summarization , 2010, IEEE Transactions on Multimedia.

[3]  Majid Mirmehdi,et al.  A shortest path representation for video summarisation , 2003, 12th International Conference on Image Analysis and Processing, 2003.Proceedings..

[4]  Ba Tu Truong,et al.  Video abstraction: A systematic review and classification , 2007, TOMCCAP.

[5]  Chao Chen,et al.  Integration of global and local information in videos for key frame extraction , 2010, 2010 IEEE International Conference on Information Reuse & Integration.

[6]  Jenq-Neng Hwang,et al.  Object-based video abstraction for video surveillance systems , 2002, IEEE Trans. Circuits Syst. Video Technol..

[7]  John R. Kender,et al.  Optimization Algorithms for the Selection of Key Frame Sequences of Variable Length , 2002, ECCV.

[8]  R. Narasimha,et al.  Key frame extraction using MPEG-7 motion descriptors , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[9]  Xian-Sheng Hua,et al.  To learn representativeness of video frames , 2005, MULTIMEDIA '05.

[10]  Qi Tian,et al.  Multilevel video representation with application to keyframe extraction , 2004, 10th International Multimedia Modelling Conference, 2004. Proceedings..

[11]  Paul Over,et al.  Evaluation campaigns and TRECVid , 2006, MIR '06.

[12]  Haibin Liu,et al.  Video linkage: group based copied video detection , 2008, CIVR '08.

[13]  Sang Uk Lee,et al.  Efficient video indexing scheme for content-based retrieval , 1999, IEEE Trans. Circuits Syst. Video Technol..

[14]  Tie-Yan Liu,et al.  Dynamic selection and effective compression of key frames for video abstraction , 2003, Pattern Recognit. Lett..

[15]  Tianming Liu,et al.  A novel video key-frame-extraction algorithm based on perceived motion energy model , 2003, IEEE Trans. Circuits Syst. Video Technol..

[16]  Xiaojun Guo,et al.  Keyframe extraction based on kmeas results to adjacent DC images similarity , 2010, 2010 2nd International Conference on Signal Processing Systems.

[17]  Jonathan Foote,et al.  Discriminative techniques for keyframe selection , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[18]  Wei Zheng,et al.  Shot Boundary Detection and Keyframe Extraction Based on Scale Invariant Feature Transform , 2009, 2009 Eighth IEEE/ACIS International Conference on Computer and Information Science.

[19]  Xiaoqin Zhang,et al.  Key-frame extraction using dominant-set clustering , 2008, 2008 IEEE International Conference on Multimedia and Expo.

[20]  Benoit Huet,et al.  Automatic video summarization , 2006 .

[21]  Stephen W. Smoliar,et al.  An integrated system for content-based video retrieval and browsing , 1997, Pattern Recognit..

[22]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[23]  Guoliang Fan,et al.  Combined key-frame extraction and object-based video segmentation , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Thomas Sikora,et al.  Feature-based video key frame extraction for low quality video sequences , 2009, 2009 10th Workshop on Image Analysis for Multimedia Interactive Services.

[25]  Regunathan Radhakrishnan,et al.  Motion activity-based extraction of key-frames from video shots , 2002, Proceedings. International Conference on Image Processing.

[26]  Mubarak Shah,et al.  Scene detection in Hollywood movies and TV shows , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[27]  Yang Peng,et al.  Behavior key frame extraction using invariant moment and unsupervised clustering , 2008, 2008 International Conference on Machine Learning and Cybernetics.

[28]  Maria Chatzigiorgaki,et al.  Real-time keyframe extraction towards video content identification , 2009, 2009 16th International Conference on Digital Signal Processing.

[29]  Avideh Zakhor,et al.  Fast similarity search and clustering of video sequences on the world-wide-web , 2005, IEEE Transactions on Multimedia.

[30]  Andreas Girgensohn,et al.  Time-Constrained Keyframe Selection Technique , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[31]  In So Kweon,et al.  A New Technique for Shot Detection and Key Frames Selection in Histogram Space , 2000 .

[32]  Anil K. Jain.  Data clustering: 50 years beyond K-means , 2008, Pattern Recognit. Lett..

[33]  Lin Honghua,et al.  Key frame extraction based on multi-scale phase-based local features , 2008, 2008 9th International Conference on Signal Processing.

[34]  Coskun Bayrak,et al.  Shot boundary detection and key frame extraction using salient region detection and structural similarity , 2010, ACM SE '10.

[35]  Qi Tian,et al.  Content-adaptive digital music watermarking based on music structure analysis , 2007, TOMCCAP.