DFP-ALC: Automatic video summarization using Distinct Frame Patch index and Appearance based Linear Clustering

Abstract Video summarization aims to create a succinct representation of videos for efficient browsing and retrieval. We propose an innovative method for the task. It includes two main steps: (i) the first step proposes a Distinct Frame Patch (DFP) index for selecting a set of good candidate frames, and (ii) the second step proposes a novel Appearance based Linear Clustering (ALC) to refine them for distinct ones. While the first step measures the content of frames, the second step considers to what extent one frame is different from another in both the spatial and temporal spaces. The experiments are performed over two publicly accessible datasets. The results show the effectiveness and efficiency of the proposed method when compared with other state-of-the-art techniques.

[1]  Pascal Fua,et al.  SLIC Superpixels Compared to State-of-the-Art Superpixel Methods , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Chinh T. Dang,et al.  RPCA-KFE: Key Frame Extraction for Video Using Robust Principal Component Analysis , 2014, IEEE Transactions on Image Processing.

[3]  Yonghuai Liu,et al.  A pertinent evaluation of automatic video summary , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[4]  Arnaldo de Albuquerque Araújo,et al.  VSUMM: A mechanism designed to produce static video summaries and a novel evaluation method , 2011, Pattern Recognit. Lett..

[5]  Yueting Zhuang,et al.  Adaptive key frame extraction using unsupervised clustering , 1998, Proceedings 1998 International Conference on Image Processing. ICIP98 (Cat. No.98CB36269).

[6]  Michael Lam,et al.  Unsupervised Video Summarization with Adversarial LSTM Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Luc Van Gool,et al.  Viewpoint-Aware Video Summarization , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[8]  Marco Pellegrini,et al.  STIMO: STIll and MOving video storyboard for the web scenario , 2009, Multimedia Tools and Applications.

[9]  Márcio Portes de Albuquerque,et al.  Image thresholding using Tsallis entropy , 2004, Pattern Recognit. Lett..

[10]  Ke Zhang,et al.  Retrospective Encoders for Video Summarization , 2018, ECCV.

[11]  Patrick Pérez,et al.  Rapid Summarisation and Browsing of Video Sequences , 2002, BMVC.

[12]  Yelena Yesha,et al.  Keyframe-based video summarization using Delaunay clustering , 2006, International Journal on Digital Libraries.

[13]  Ke Zhang,et al.  Video Summarization with Long Short-Term Memory , 2016, ECCV.

[14]  Jiebo Luo,et al.  Towards Extracting Semantically Meaningful Key Frames From Personal Video Clips: From Humans to Computers , 2009, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Ba Tu Truong,et al.  Video abstraction: A systematic review and classification , 2007, TOMCCAP.

[16]  Chinh T. Dang,et al.  Heterogeneity Image Patch Index and Its Application to Consumer Video Summarization , 2014, IEEE Transactions on Image Processing.

[17]  Jianmin Jiang,et al.  A novel clustering method for static video summarization , 2017, Multimedia Tools and Applications.

[18]  Ke Zhang,et al.  Summary Transfer: Exemplar-Based Subset Selection for Video Summarization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).