OPFSumm: on the video summarization using Optimum-Path Forest

Video summarization attempts at encoding a given video into a compact representation for a better storage and retrieval purposes. This work copes with the problem of static video summarization using the unsupervised Optimum-Path Forest (OPF). We sampled the encoded video sequence into frames and extracted features based on color information or spectral properties. After that, meaningless frames are removed, and OPF models the problem of video summarization as a clustering process. Possible redundant keyframes are filtered, and at last the video summary is created based on non-redundant keyframes. We presented a more in-depth study that also considers temporal information to obtain better video representations. The experiments over three public datasets were analyzed through F -measure evaluation metric and showed the robustness of OPF for automatic video summarization: 0.19 for SumMe dataset, 0.728 concerning Open Video dataset, and 0.451 regarding YouTube dataset..

[1]  João Paulo Papa,et al.  On the Training of Artificial Neural Networks with Radial Basis Function Using Optimum-Path Forest Clustering , 2014, 2014 22nd International Conference on Pattern Recognition.

[2]  João Paulo Papa,et al.  Supervised pattern classification based on optimum‐path forest , 2009, Int. J. Imaging Syst. Technol..

[3]  Dim P. Papadopoulos,et al.  Video Summarization Using a Self-Growing and Self-Organized Neural Gas Network , 2011, MIRAGE.

[4]  Luc Van Gool,et al.  Video summarization by learning submodular mixtures of objectives , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  YoungSik Choi,et al.  Video Summarization Using Fuzzy One-Class Support Vector Machine , 2004, ICCSA.

[6]  Ramin Zabih,et al.  Comparing images using color coherence vectors , 1997, MULTIMEDIA '96.

[7]  João Paulo Papa,et al.  Fine-tuning Deep Belief Networks using Harmony Search , 2016, Appl. Soft Comput..

[8]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[9]  Jiasong Zhu,et al.  Learning deep semantic attributes for user video summarization , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[10]  Jurandy Almeida,et al.  Static Video Summarization through Optimum-Path Forest Clustering , 2014, CIARP.

[11]  João Paulo Papa,et al.  Optimum-Path Forest based on k-connectivity: Theory and applications , 2017, Pattern Recognit. Lett..

[12]  Luc Van Gool,et al.  Creating Summaries from User Videos , 2014, ECCV.

[13]  Jorge Stolfi,et al.  IFTrace: Video segmentation of deformable objects using the Image Foresting Transform , 2012, Comput. Vis. Image Underst..

[14]  Jurandy Almeida,et al.  Online video summarization on compressed domain , 2013, J. Vis. Commun. Image Represent..

[15]  Ke Zhang,et al.  Video Summarization with Long Short-Term Memory , 2016, ECCV.

[16]  João Paulo Papa,et al.  A Learning Algorithm for the Optimum-Path Forest Classifier , 2009, GbRPR.

[17]  João Paulo Papa,et al.  Deep Boltzmann machines for robust fingerprint spoofing attack detection , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[18]  Jing Huang,et al.  Image indexing using color correlograms , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Jurandy Almeida,et al.  Temporal-and Spatial-Driven Video Summarization Using Optimum-Path Forest , 2016, 2016 29th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI).

[20]  Michael Lam,et al.  Unsupervised Video Summarization with Adversarial LSTM Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Mario A. Nascimento,et al.  A compact and efficient image retrieval approach based on border/interior pixel classification , 2002, CIKM '02.

[22]  João Paulo Papa,et al.  Efficient supervised optimum-path forest classification for large datasets , 2012, Pattern Recognit..

[23]  João Paulo Papa,et al.  Improving semi-supervised learning through optimum connectivity , 2016, Pattern Recognit..

[24]  David Salesin,et al.  Fast multiresolution image querying , 1995, SIGGRAPH.

[25]  Yelena Yesha,et al.  Keyframe-based video summarization using Delaunay clustering , 2006, International Journal on Digital Libraries.

[26]  Adriano M. Pereira,et al.  A video summarization approach based on the emulation of bottom-up mechanisms of visual attention , 2017, Journal of Intelligent Information Systems.

[27]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Luming Zhang,et al.  An Effective Video Summarization Framework Toward Handheld Devices , 2015, IEEE Transactions on Industrial Electronics.

[29]  Jurandy Almeida,et al.  VISON: VIdeo Summarization for ONline applications , 2012, Pattern Recognit. Lett..

[30]  Kristen Grauman,et al.  Diverse Sequential Subset Selection for Supervised Video Summarization , 2014, NIPS.

[31]  Ke Zhang,et al.  Summary Transfer: Exemplar-Based Subset Selection for Video Summarization , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Guojun Lu,et al.  Shape-based image retrieval using generic Fourier descriptor , 2002, Signal Process. Image Commun..

[33]  Marcelo Mendoza,et al.  Progress in Pattern Recognition, Image Analysis, Computer Vision, and Applications , 2017, Lecture Notes in Computer Science.

[34]  Marco Pellegrini,et al.  STIMO: STIll and MOving video storyboard for the web scenario , 2009, Multimedia Tools and Applications.

[35]  Harry W. Agius,et al.  Video summarisation: A conceptual framework and survey of the state of the art , 2008, J. Vis. Commun. Image Represent..

[36]  Alexandre X. Falcão,et al.  Data clustering as an optimum‐path forest problem with applications in image analysis , 2009, Int. J. Imaging Syst. Technol..

[37]  Rodrigo Nakamura,et al.  Toward Satellite-Based Land Cover Classification Through Optimum-Path Forest , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[38]  João Paulo Papa,et al.  A New Variant of the Optimum-Path Forest Classifier , 2008, ISVC.

[39]  Arnaldo de Albuquerque Araújo,et al.  VSUMM: A mechanism designed to produce static video summaries and a novel evaluation method , 2011, Pattern Recognit. Lett..

[40]  David S. Doermann,et al.  Video summarization by curve simplification , 1998, MULTIMEDIA '98.

[41]  Ananda S. Chowdhury,et al.  Scalable Video Summarization Using Skeleton Graph and Random Walk , 2014, 2014 22nd International Conference on Pattern Recognition.