Surveillance video synopsis in the compressed domain for fast video browsing

The traditional pixel-domain based video analysis methods have taken dominated places for long. However, due to the rapidly increasing volume and resolution of surveillance video, the desirable fast and scalable browsing encounters significant challenges in terms of efficiency and flexibility. Under this circumstance, operating surveillance video in compressed domain has aroused great concern in academy and industry. In order to perform the intelligent video analysis task on the premise of preserving accuracy and controlling complexity, this paper presents a compressed-domain approach for massive surveillance video synopsis generation, labeling and browsing. The main work and achievements include: (1) a compressed-domain scheme is established to condense the compressed surveillance video and record the synopsis results; (2) a background modeling method via the Motion Vector based Local Binary Pattern (MVLBP) is introduced to extract moving objects in an efficient way; (3) an object flags based synopsis labeling method is proposed to represent the object regions as well as their display modes in a flexible way. Experimental results show that the video analysis system based on this framework can provide not only efficient synopsis generation but also flexible scalable or playback browsing.

[1]  J. Koenderink.  Q… , 2014, Les noms officiels des communes de Wallonie, de Bruxelles-Capitale et de la communaute germanophone.

[2]  Yael Pritch,et al.  Making a Long Video Short: Dynamic Video Synopsis , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[3]  Wei Tsang Ooi,et al.  Crowdsourced automatic zoom and scroll for video retargeting , 2010, ACM Multimedia.

[4]  Deb Roy,et al.  An immersive system for browsing and visualizing surveillance video , 2010, ACM Multimedia.

[5]  Tieniu Tan,et al.  Multi-thread Parsing for Recognizing Complex Events in Videos , 2008, ECCV.

[6]  Bernd Girod,et al.  Analysis of Packet Loss for Compressed Video: Effect of Burst Losses and Correlation Between Error Frames , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Wen Gao,et al.  Modeling Background and Segmenting Moving Objects from Compressed Video , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[8]  Yannis Avrithis,et al.  Efficient content representation in MPEG video databases , 1998, Proceedings. IEEE Workshop on Content-Based Access of Image and Video Libraries (Cat. No.98EX173).

[9]  Y. Bar-Shalom.  Tracking and data association , 1988 .

[10]  Ba Tu Truong,et al.  Video abstraction: A systematic review and classification , 2007, TOMCCAP.

[11]  Hanan Samet,et al.  Data structures for quadtree approximation and compression , 1985, CACM.

[12]  Anni Cai,et al.  A surveillance video analysis and storage scheme for scalable synopsis browsing , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[13]  Bu-Sung Lee,et al.  Explore and Model Better I-Frames for Video Coding , 2011, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Jurandy Almeida,et al.  Online video summarization on compressed domain , 2013, J. Vis. Commun. Image Represent..

[15]  Mubarak Shah,et al.  Automated Visual Surveillance in Realistic Scenarios , 2007, IEEE MultiMedia.

[16]  A. Yildiz,et al.  Fast non-linear video synopsis , 2008, 2008 23rd International Symposium on Computer and Information Sciences.

[17]  Matti Pietikäinen,et al.  Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Kiyoharu Aizawa,et al.  Interactive retrieval for multi-camera surveillance systems featuring spatio-temporal summarization , 2008, ACM Multimedia.

[19]  Sung Wook Baik,et al.  Adaptive key frame extraction for video summarization using an aggregation mechanism , 2012, J. Vis. Commun. Image Represent..

[20]  Xuelong Li,et al.  Biologically Inspired Features for Scene Classification in Video Surveillance , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.

[22]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[23]  Matti Pietikäinen,et al.  Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Peter Lambert,et al.  Moving object detection in the H.264/AVC compressed domain for video surveillance applications , 2009, J. Vis. Commun. Image Represent..

[26]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[27]  Takeo Kanade,et al.  Video skimming and characterization through the combination of image and language understanding , 1998, Proceedings 1998 IEEE International Workshop on Content-Based Access of Image and Video Database.

[28]  Zhu Li,et al.  Real-time human action recognition by luminance field trajectory analysis , 2008, ACM Multimedia.

[29]  Parvaneh Saeedi,et al.  Moving Region Segmentation From Compressed Video Using Global Motion Estimation and Markov Random Fields , 2011, IEEE Transactions on Multimedia.

[30]  Chong-Wah Ngo,et al.  Automatic video summarization by graph modeling , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[31]  Xianguo Zhang,et al.  Low-complexity and high-efficiency background modeling for surveillance video coding , 2012, 2012 Visual Communications and Image Processing.

[32]  Shengcai Liao,et al.  Online Principal Background Selection for Video Synopsis , 2010, 2010 20th International Conference on Pattern Recognition.

[33]  Bu-Sung Lee,et al.  Video coding with dynamic background , 2013, EURASIP J. Adv. Signal Process..

[34]  Xuelong Li,et al.  Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Terrance E. Boult,et al.  Into the woods: visual surveillance of noncooperative and camouflaged targets in complex outdoor settings , 2001, Proc. IEEE.

[36]  HyunWook Park,et al.  Moving-object segmentation in compressed video , 2001 .

[37]  Wu-chi Feng,et al.  Supporting region-of-interest cropping through constrained compression , 2008, ACM Multimedia.

[38]  R. Venkatesh Babu,et al.  Video object segmentation: a compressed domain approach , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[39]  Yael Pritch,et al.  This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, 2008 1 Non-Chronological Video , 2022 .

[40]  Wen Gao,et al.  Robust moving object segmentation on H.264/AVC compressed video using the block-based MRF model , 2005, Real Time Imaging.

[41]  Ming-Ting Sun,et al.  Automatic video activity detection using compressed domain motion trajectories for H.264 videos , 2011, J. Vis. Commun. Image Represent..

[42]  Rama Chellappa,et al.  An ontology based approach for activity recognition from video , 2008, ACM Multimedia.