Intelligent video surveillance for real time energy savings in smart buildings using HEVC compressed domain features

This work presents a unique approach to utilize the existing video surveillance infrastructure to optimize electricity consumption in large indoor spaces such as library reading halls, waiting rooms, indoor sports complex, large dormitories etc. The proposed method extracts features from the encoder loop of the digital video recorder (DVR) to analyze the foreground activity. High Efficiency Video Coding (HEVC) is chosen as the target video codec, as it is going to be the most widely deployed video codec in the future DVRs. The bit rate efficiency of HEVC is almost double in comparison to H.264/AVC for same video quality [17]. The proposed method utilizes only compressed domain parameters from video stream such as motion vectors clustering information, block partitioning modes etc., which allows to develop algorithms with relatively lower computational complexity and memory requirement than pixel based methods thus achieving real time performance.

[1]  Marc Van Droogenbroeck,et al.  ViBE: A powerful random technique to estimate the background in video sequences , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[2]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[3]  Zulin Wang,et al.  Learning to segment videos in HEVC compressed domain , 2015, 2015 International Conference on Wireless Communications & Signal Processing (WCSP).

[4]  Fatih Murat Porikli,et al.  A Novel Video Dataset for Change Detection Benchmarking , 2014, IEEE Transactions on Image Processing.

[5]  David Suter,et al.  Background Subtraction Based on a Robust Consensus Method , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[6]  Gary J. Sullivan,et al.  High Efficiency Video Coding (HEVC), Algorithms and Architectures , 2014, Integrated Circuits and Systems.

[7]  Bob Zhang,et al.  Background modeling methods in video analysis: A review and comparative evaluation , 2016, CAAI Trans. Intell. Technol..

[8]  Kai Chen,et al.  Moving Object Counting Using a Tripwire in H.265/HEVC Bitstreams for Video Surveillance , 2016, IEEE Access.

[9]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[10]  R. Venkatesh Babu,et al.  A survey on compressed domain video analysis techniques , 2014, Multimedia Tools and Applications.

[11]  Wen Gao,et al.  Modeling Background and Segmenting Moving Objects from Compressed Video , 2008, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Tiejun Huang,et al.  Representing Visual Objects in HEVC Coding Loop , 2014, IEEE Journal on Emerging and Selected Topics in Circuits and Systems.

[13]  Ming Yang,et al.  A rapid abnormal event detection method for surveillance video based on a novel feature in compressed domain of HEVC , 2014, 2014 IEEE International Conference on Multimedia and Expo (ICME).

[14]  Gary J. Sullivan,et al.  Overview of the High Efficiency Video Coding (HEVC) Standard , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  KokSheik Wong,et al.  Moving object detection in HEVC video by frame sub-sampling , 2015, 2015 International Symposium on Intelligent Signal Processing and Communication Systems (ISPACS).

[16]  Zoran Zivkovic,et al.  Improved adaptive Gaussian mixture model for background subtraction , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[17]  Debin Zhao,et al.  Real-Time Moving Object Segmentation and Classification From HEVC Compressed Surveillance Video , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[18]  Gerhard Rigoll,et al.  Background segmentation with feedback: The Pixel-Based Adaptive Segmenter , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[19]  Larry S. Davis,et al.  Background modeling and subtraction by codebook construction , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[20]  Malay Kumar Kundu,et al.  Efficient Foreground Extraction From HEVC Compressed Video for Application to Real-Time Analysis of Surveillance ‘Big’ Data , 2015, IEEE Transactions on Image Processing.

[21]  Lucia Maddalena,et al.  A Self-Organizing Approach to Background Subtraction for Visual Surveillance Applications , 2008, IEEE Transactions on Image Processing.