Compressed-domain Video Synopsis via 3D Graph Cut and Blank Frame Deletion

With the wide adoption in surveillance video analysis, some pixel-domain analysis methods are not only used in academia but are also garnering the attention of industry. However, because of the increasing data volume of video, how to analyze and browse surveillance videos in a fast and effective way has continued to present an intractable problem in practical applications. Therefore, a synopsis analysis method based on blank frame deletion in the compressed domain is proposed in this paper to achieve fast synopsis analysis and video browsing. Our main contributions are: 1) An algorithm of compressed-domain 3D graph cut is introduced to support the fast object tubes segmentation and synopsis analysis in the compressed domain. 2) A concept, referred to as virtual blank frame, is introduced and applied into synopsis analysis. This proposed video synopsis method can further synopsize the video in the spatio-temporal domain through deleting the virtual blank frames after real blank frame deletion. 3) A comprehensive solution, synopsis analyzing based on blank frame deletion in the compressed domain and fast browsing synopsis video in the pixel domain, is provided for video synopsis, which is effective and efficient for video surveillance. Experimental results show that our approach maintains a stable performance in fast synopsis analysis in the H.264 compressed domain.

[1]  Rui-min Hu,et al.  Surveillance video synopsis in the compressed domain for fast video browsing , 2013, J. Vis. Commun. Image Represent..

[2]  Yael Pritch,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2008 1 Non-Chronological Video , 2022 .

[3]  Yael Pritch,et al.  Making a Long Video Short: Dynamic Video Synopsis , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[4]  Heguang Liu,et al.  A novel scheme to code object flags for video synopsis , 2012, 2012 Visual Communications and Image Processing.

[5]  Stan Z. Li,et al.  Online content-aware video condensation , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Wen Gao,et al.  Robust moving object segmentation on H.264/AVC compressed video using the block-based MRF model , 2005, Real Time Imaging.

[7]  Shizheng Wang,et al.  Wireless Video Surveillance System Based on Incremental Learning Face Detection , 2015, MMM.

[8]  Ruimin Hu,et al.  Fast Synopsis for Moving Objects Using Compressed Video , 2014, IEEE Signal Processing Letters.

[9]  R. Venkatesh Babu,et al.  Video object segmentation: a compressed domain approach , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Vladimir Kolmogorov,et al.  An experimental comparison of min-cut/max- flow algorithms for energy minimization in vision , 2001, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Xuelong Li,et al.  Surveillance Video Synopsis via Scaling Down Objects , 2016, IEEE Transactions on Image Processing.

[12]  Xiaolong Wang,et al.  Background modeling using Local Binary Patterns Of Motion Vector , 2012, 2012 Visual Communications and Image Processing.

[13]  Chun-Rong Huang,et al.  Maximum a Posteriori Probability Estimation for Online Surveillance Video Synopsis , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Yue Wang,et al.  Motion-State-Adaptive Video Summarization via Spatiotemporal Analysis , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  R. Venkatesh Babu,et al.  A survey on compressed domain video analysis techniques , 2014, Multimedia Tools and Applications.

[16]  Shengcai Liao,et al.  High-Performance Video Condensation System , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[17]  Parvaneh Saeedi,et al.  Moving Region Segmentation From Compressed Video Using Global Motion Estimation and Markov Random Fields , 2011, IEEE Transactions on Multimedia.

[18]  Zhiwei Huang,et al.  Moving Objects Segmentation from compressed surveillance video based on Motion Estimation , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[19]  Yongwei Nie,et al.  Compact Video Synopsis via Global Spatiotemporal Optimization , 2013, IEEE Transactions on Visualization and Computer Graphics.

[20]  Zhongyuan Wang,et al.  Demo paper: Video retrieval synopsis for moving objects , 2013, 2013 IEEE International Conference on Multimedia and Expo Workshops (ICMEW).

[21]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  David Suter,et al.  A consensus-based method for tracking: Modelling background scenario and foreground appearance , 2007, Pattern Recognit..

[23]  S. Avidan,et al.  Seam carving for content-aware image resizing , 2007, SIGGRAPH 2007.

[24]  Anni Cai,et al.  A surveillance video analysis and storage scheme for scalable synopsis browsing , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[25]  Nikos Paragios,et al.  Background modeling and subtraction of dynamic scenes , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[26]  Chuohao Yeo,et al.  Compressed domain video processing of meetings for activity estimation in dominance classification and slide transition detection , 2008 .

[27]  Yi He,et al.  Fast Online Video Synopsis Based on Potential Collision Graph , 2017, IEEE Signal Processing Letters.

[28]  VekslerOlga,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001 .

[29]  Seung-Won Jung,et al.  Order-Preserving Condensation of Moving Objects in Surveillance Videos , 2016, IEEE Transactions on Intelligent Transportation Systems.

[30]  Tao Mei,et al.  Near-lossless semantic video summarization and its applications to video analysis , 2013, TOMCCAP.

[31]  Chien-Li Chou,et al.  Coherent event-based surveillance video synopsis using trajectory clustering , 2015, 2015 IEEE International Conference on Multimedia & Expo Workshops (ICMEW).

[32]  Bo Yan,et al.  An Effective Video Synopsis Approach with Seam Carving , 2016, IEEE Signal Processing Letters.

[33]  Janusz Konrad,et al.  Video Condensation by Ribbon Carving , 2009, IEEE Transactions on Image Processing.

[34]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.