Content Coverage and Redundancy Removal in Video Summarization

Over the past decade, research in the field of Content-Based Video Retrieval Systems (CBVRS) has attracted much attention as it encompasses processing of all the other media types i.e. text, image and audio. Video summarization is one of the most important applications as it potentially enables efficient and faster browsing of large video collections. A concise version of the video is often required due to constraints in viewing time, storage, communication bandwidth as well as power. Thus, the task of video summarization is to effectively extract the most important portions of the video, without sacrificing the semantic information in it. The results of video summarization can be used in many CBVRS applications like semantic indexing, video surveillance copied video detection etc. However, the quality of the summarization task depends on two basic aspects: content coverage and redundancy removal. These two aspects are both important and contradictory to each other. This chapter aims to provide an insight into the state-of-the-art approaches used for this booming field of research.

[1]  Patrick Gros,et al.  A Geometrical Key-Frame Selection Method Exploiting Dominant Motion Estimation in Video , 2004, CIVR.

[2]  Cyrus Shahabi,et al.  Key Frame Selection Algorithms for Automatic Generation of Panoramic Images from Crowdsourced Geo-tagged Videos , 2014, W2GIS.

[3]  Guillermo Cámara Chávez,et al.  A New Method for Static Video Summarization Using Local Descriptors and Video Temporal Segmentation , 2013, 2013 XXVI Conference on Graphics, Patterns and Images.

[4]  R. Narasimha,et al.  Key frame extraction using MPEG-7 motion descriptors , 2003, The Thrity-Seventh Asilomar Conference on Signals, Systems & Computers, 2003.

[5]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[6]  B. B. Meshram,et al.  Content based video retrieval systems , 2012, ArXiv.

[7]  Yo-Sung Ho,et al.  Content-based event retrieval using semantic scene interpretation for automated traffic surveillance , 2001, IEEE Trans. Intell. Transp. Syst..

[8]  Michael R. Lyu,et al.  Video summarization by video structure analysis and graph optimization , 2004, 2004 IEEE International Conference on Multimedia and Expo (ICME) (IEEE Cat. No.04TH8763).

[9]  HongJiang Zhang,et al.  Motion texture: a new motion based video representation , 2002, Object recognition supported by user interaction for service robots.

[10]  S. Thiruchadai Pandeeswari,et al.  VISUAL ATTENTION BASED KEYFRAMES EXTRACTION AND VIDEO SUMMARIZATION , 2012 .

[11]  Shogo Muramatsu,et al.  Video key frame selection by clustering wavelet coefficients , 2004, 2004 12th European Signal Processing Conference.

[12]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[13]  Andreas Girgensohn,et al.  Time-Constrained Keyframe Selection Technique , 1999, Proceedings IEEE International Conference on Multimedia Computing and Systems.

[14]  Rong Yan,et al.  A review of text and image retrieval approaches for broadcast news video , 2007, Information Retrieval.

[15]  Masaharu Ogawa,et al.  A highlight scene detection and video summarization system using audio feature for a personal video recorder , 2005, IEEE Transactions on Consumer Electronics.

[16]  Alan F. Smeaton,et al.  TRECVID 2004 Experiments in Dublin City University , 2004, TRECVID.

[17]  Matthieu Cord,et al.  Rushes summarization by IRIM consortium: redundancy removal and multi-feature fusion , 2008, TVS '08.

[18]  Sang Uk Lee,et al.  Efficient video indexing scheme for content-based retrieval , 1999, IEEE Trans. Circuits Syst. Video Technol..

[19]  Yelena Yesha,et al.  Keyframe-based video summarization using Delaunay clustering , 2006, International Journal on Digital Libraries.

[20]  Ting Wang,et al.  An Approach to Video Key-frame Extraction Based on Rough Set , 2007, 2007 International Conference on Multimedia and Ubiquitous Engineering (MUE'07).

[21]  Ahmed M. Elgammal,et al.  Information Theoretic Key Frame Selection for Action Recognition , 2008, BMVC.

[22]  Santosh S. Vempala,et al.  Latent semantic indexing: a probabilistic analysis , 1998, PODS '98.

[23]  Vincent Lepetit,et al.  DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Chong-Wah Ngo,et al.  Video summarization and scene detection by graph modeling , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[25]  Shih-Fu Chang,et al.  Motion trajectory matching of video objects , 1999, Electronic Imaging.

[26]  Shingo Uchihashi,et al.  Video Manga: generating semantically meaningful video summaries , 1999, MULTIMEDIA '99.

[27]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[28]  Xin Liu,et al.  Video summarization with minimal visual content redundancies , 2001, Proceedings 2001 International Conference on Image Processing (Cat. No.01CH37205).

[29]  Janko Calic,et al.  Efficient key-frame extraction and video analysis , 2002, Proceedings. International Conference on Information Technology: Coding and Computing.

[30]  Guoliang Fan,et al.  Joint Key-Frame Extraction and Object-Based Video Segmentation , 2005, 2005 Seventh IEEE Workshops on Applications of Computer Vision (WACV/MOTION'05) - Volume 1.

[31]  Kuo-Chin Fan,et al.  Motion Flow-Based Video Retrieval , 2007, IEEE Transactions on Multimedia.

[32]  Yueting Zhuang,et al.  Adaptive key frame extraction using unsupervised clustering , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[33]  Nicu Sebe,et al.  Object Recognition for Video Retrieval , 2002, CIVR.

[34]  Jonathan Foote,et al.  Discriminative techniques for keyframe selection , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[35]  Duy-Dinh Le,et al.  Face Retrieval in Broadcasting News Video by Fusing Temporal and Intensity Information , 2006, CIVR.

[36]  Alberto Del Bimbo,et al.  Symbolic Description and Visual Querying of Image Sequences Using Spatio-Temporal Logic , 1995, IEEE Trans. Knowl. Data Eng..

[37]  Tie-Yan Liu,et al.  Dynamic selection and effective compression of key frames for video abstraction , 2003, Pattern Recognit. Lett..

[38]  Andrew Zisserman,et al.  Person Spotting: Video Shot Retrieval for Face Sets , 2005, CIVR.

[39]  Michael R. Lyu,et al.  Video summarization by spatial-temporal graph optimization , 2004, 2004 IEEE International Symposium on Circuits and Systems (IEEE Cat. No.04CH37512).

[40]  Dipti Prasad Mukherjee,et al.  Key Frame Estimation in Video Using Randomness Measure of Feature Point Pattern , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[41]  Siddhartha Bhattacharyya,et al.  Towards redundancy reduction in storyboard representation for static video summarization , 2014, 2014 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[42]  Dan Schonfeld,et al.  Real-Time Motion Trajectory-Based Indexing and Retrieval of Video Sequences , 2007, IEEE Transactions on Multimedia.

[43]  Jeho Nam,et al.  Dynamic video summarization and visualization , 1999, MULTIMEDIA '99.

[44]  Alan Hanjalic,et al.  An integrated scheme for automated video abstraction based on unsupervised cluster-validity analysis , 1999, IEEE Trans. Circuits Syst. Video Technol..

[45]  Jana Machajdik,et al.  A Keyframe Selection of Lifelog Image Sequences , 2013, MVA.

[46]  Georgios Tziritas,et al.  Equivalent Key Frames Selection Based on Iso-Content Principles , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[47]  Ba Tu Truong,et al.  Video abstraction: A systematic review and classification , 2007, TOMCCAP.

[48]  George Economou,et al.  Key frame extraction in video sequences: a vantage points approach , 2007, 2007 IEEE 9th Workshop on Multimedia Signal Processing.

[49]  Boon-Lock Yeo,et al.  Video visualization for compact presentation and fast browsing of pictorial content , 1997, IEEE Trans. Circuits Syst. Video Technol..

[50]  John R. Smith,et al.  Multimedia semantic indexing using model vectors , 2003, 2003 International Conference on Multimedia and Expo. ICME '03. Proceedings (Cat. No.03TH8698).

[51]  Siddhartha Bhattacharyya,et al.  Enhancement of perceptual quality in static video summarization using minimal spanning tree approach , 2015, 2015 IEEE International Conference on Signal Processing, Informatics, Communication and Energy Systems (SPICES).

[52]  Katsumi Tanaka,et al.  Querying Video Data by Spatio-Temporal Relationships of Moving Object Traces , 2002, VDB.

[53]  Xavier Binefa,et al.  An EM algorithm for video summarization, generative model approach , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[54]  Patrick Pérez,et al.  Nonparametric motion characterization using causal probabilistic models for video indexing and retrieval , 2002, IEEE Trans. Image Process..

[55]  Haibin Liu,et al.  Video linkage: group based copied video detection , 2008, CIVR '08.

[56]  HongJiang Zhang,et al.  A model of motion attention for video skimming , 2002, Proceedings. International Conference on Image Processing.

[57]  Antonio Torralba,et al.  Modeling the Shape of the Scene: A Holistic Representation of the Spatial Envelope , 2001, International Journal of Computer Vision.

[58]  Siddhartha Bhattacharyya,et al.  Video Shot Segmentation Using Spatio-temporal Fuzzy Hostility Index and Automatic Threshold , 2014, 2014 Fourth International Conference on Communication Systems and Network Technologies.

[59]  A. Lakshmi,et al.  Faculty Perception Towards Institutional Climate with Special Reference to Namakkal District - An Empirical Study , 2012 .

[60]  John R. Smith,et al.  IBM Research TRECVID-2009 Video Retrieval System , 2009, TRECVID.

[61]  Xin Liu,et al.  Generating optimal video summaries , 2000, 2000 IEEE International Conference on Multimedia and Expo. ICME2000. Proceedings. Latest Advances in the Fast Changing World of Multimedia (Cat. No.00TH8532).

[62]  Xin Liu,et al.  Video summarization and retrieval using singular value decomposition , 2003, Multimedia Systems.

[63]  David Doermann,et al.  Video indexing and retrieval based on recognized text , 2002, 2002 IEEE Workshop on Multimedia Signal Processing..

[64]  Bernard Mérialdo,et al.  Sequence alignment for redundancy removal in video rushes summarization , 2008, TVS '08.

[65]  Michael A. Smith,et al.  Video skimming and characterization through the combination of image and language understanding techniques , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[66]  Nathalie Guyader,et al.  Video Summarization Based on Camera Motion and a Subjective Evaluation Method , 2007, EURASIP J. Image Video Process..

[67]  Guoliang Fan,et al.  Combined key-frame extraction and object-based video segmentation , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[68]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[69]  Andrian Marcus,et al.  Recovering documentation-to-source-code traceability links using latent semantic indexing , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[70]  Chao Chen,et al.  Integration of global and local information in videos for key frame extraction , 2010, 2010 IEEE International Conference on Information Reuse & Integration.

[71]  David S. Doermann,et al.  Video summarization by curve simplification , 1998, MULTIMEDIA '98.

[72]  Anoop Gupta,et al.  Auto-summarization of audio-video presentations , 1999, MULTIMEDIA '99.

[73]  Chia-Hung Yeh,et al.  Techniques for movie content analysis and skimming: tutorial and overview on video abstraction techniques , 2006, IEEE Signal Processing Magazine.

[74]  Shiyang Lu,et al.  Keypoint-Based Keyframe Selection , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[75]  Kin-Man Lam,et al.  A new key frame representation for video segment retrieval , 2005, IEEE Transactions on Circuits and Systems for Video Technology.

[76]  Shih-Fu Chang,et al.  A fully automated content-based video search engine supporting spatiotemporal queries , 1998, IEEE Trans. Circuits Syst. Video Technol..

[77]  Changsheng Xu,et al.  Automatic mobile sports highlights , 2005, 2005 IEEE International Conference on Multimedia and Expo.

[78]  Maria Chatzigiorgaki,et al.  Real-time keyframe extraction towards video content identification , 2009, 2009 16th International Conference on Digital Signal Processing.

[79]  Nuno Vasconcelos,et al.  A spatiotemporal motion model for video summarization , 1998, Proceedings. 1998 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No.98CB36231).

[80]  Rainer Lienhart Dynamic video summarization of home video , 1999, Electronic Imaging.