Shot based keyframe extraction for ecological video indexing and retrieval

Abstract Among possible research area in multimedia, keyframe extraction is an important topic that provides video summarization, faster browsing and accessing of wide video collections. In this paper, we propose a new automatic shot based keyframe extraction for video indexing and retrieval applications. Initially, the frames are sequentially clustered into shots by using feature extraction, continuity value construction steps of shot boundary detection process and the shot frame clustering technique. The cluster having a larger dispersion rate is selected for inter cluster similarity analysis (ICSA) and the sub-shot based keyframes are extracted using ICSA. The proposed shot boundary detection algorithm and video keyframe extraction technique are implemented and evaluated on publicly available ecological video datasets. Compared with existing related algorithms, our method yields better F1-score of 94.2% for shot boundary detection and better results for keyframe extraction. The keyframes extracted by the proposed method are used for video indexing and retrieval.

[1]  Tiecheng Liu,et al.  Summarization of Visual Content in Instructional Videos , 2007, IEEE Transactions on Multimedia.

[2]  Boon-Lock Yeo,et al.  Rapid scene analysis on compressed video , 1995, IEEE Trans. Circuits Syst. Video Technol..

[3]  M. Ibrahim Sezan,et al.  A semantic event-detection approach and its application to detecting hunts in wildlife vide , 2000, IEEE Trans. Circuits Syst. Video Technol..

[4]  George Economou,et al.  Combining graph connectivity & dominant set clustering for video summarization , 2009, Multimedia Tools and Applications.

[5]  Yueting Zhuang,et al.  Adaptive key frame extraction using unsupervised clustering , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[6]  Paul Over,et al.  TREC video retrieval evaluation TRECVID , 2008 .

[7]  Thomas Mueller,et al.  Introducing AMV (Animal Movement Visualizer), a visualization tool for animal movement data from satellite collars and radiotelemetry , 2013, Ecol. Informatics.

[8]  Dong-Sik Jang,et al.  Gradual shot boundary detection using localized edge blocks , 2006, Multimedia Tools and Applications.

[9]  Rainer Lienhart,et al.  Comparison of automatic shot boundary detection algorithms , 1998, Electronic Imaging.

[10]  Whoi-Yul Kim,et al.  Automatic video summarizing tool using MPEG-7 descriptors for personal video recorder , 2003, IEEE Trans. Consumer Electron..

[11]  Andreas Girgensohn,et al.  Time-Constrained Keyframe Selection Technique , 2004, Multimedia Tools and Applications.

[12]  Chih-Jen Lin,et al.  A Study on Threshold Selection for Multi-label Classification , 2007 .

[13]  Shiguo Lian,et al.  Automatic video temporal segmentation based on multiple features , 2011, Soft Comput..

[14]  Paul Over,et al.  The trecvid 2007 BBC rushes summarization evaluation pilot , 2007, TVS '07.

[15]  Chia-Hung Yeh,et al.  Techniques for movie content analysis and skimming: tutorial and overview on video abstraction techniques , 2006, IEEE Signal Processing Magazine.

[16]  Bhabatosh Chanda,et al.  A Model-Based Shot Boundary Detection Technique Using Frame Transition Parameters , 2012, IEEE Transactions on Multimedia.

[17]  Ullas Gargi,et al.  Performance characterization of video-shot-change detection methods , 2000, IEEE Trans. Circuits Syst. Video Technol..

[18]  Tianming Liu,et al.  A novel video key-frame-extraction algorithm based on perceived motion energy model , 2003, IEEE Trans. Circuits Syst. Video Technol..

[19]  Ba Tu Truong,et al.  Video abstraction: A systematic review and classification , 2007, TOMCCAP.

[20]  Mahmood Fathy,et al.  Hierarchical Keyframe-based Video Summarization Using QR-Decomposition and Modified -Means Clustering , 2010, EURASIP J. Adv. Signal Process..

[21]  Xin Liu,et al.  Generating optimal video summaries , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[22]  Donald A. Adjeroh,et al.  Adaptive Edge-Oriented Shot Boundary Detection , 2009, EURASIP J. Image Video Process..

[23]  Willem Bouten,et al.  UvA-DARE (Digital Academic Repository) A semantically integrated, user-friendly data model for species observation data , 2012 .

[24]  Chong-Wah Ngo,et al.  Representations of Keypoint-Based Semantic Concept Detection: A Comprehensive Study , 2010, IEEE Transactions on Multimedia.

[25]  S. Domnic,et al.  Edge Strength Extraction using Orthogonal Vectors for Shot Boundary Detection , 2012 .

[26]  Yannis Avrithis,et al.  Concept detection and keyframe extraction using a visual thesaurus , 2009, Multimedia Tools and Applications.

[27]  Ioannis Pitas,et al.  Information theory-based shot cut/fade detection and video summarization , 2006, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Martin K. Purvis,et al.  Wildlife video key-frame extraction based on novelty detection in semantic context , 2011, Multimedia Tools and Applications.

[29]  Neill W. Campbell,et al.  Visual abstraction of wildlife footage using Gaussian mixture models and the minimum description length criterion , 2002, Object recognition supported by user interaction for service robots.

[30]  Tudor Barbu,et al.  Novel automatic video cut detection technique using Gabor filtering , 2009, Comput. Electr. Eng..

[31]  Tiziana D'Orazio,et al.  A review of vision-based systems for soccer video analysis , 2010, Pattern Recognit..

[32]  Raimondo Schettini,et al.  Erratum to: An innovative algorithm for key frame extraction in video summarization , 2006, Journal of Real-Time Image Processing.

[33]  Lilly Suriani Affendey,et al.  An integrated semantic-based approach in concept based video retrieval , 2011, Multimedia Tools and Applications.

[34]  John S. Boreczky,et al.  Comparison of video shot boundary detection techniques , 1996, J. Electronic Imaging.

[35]  Geoff Groom,et al.  Remote sensing image data and automated analysis to describe marine bird distributions and abundances , 2013, Ecol. Informatics.