Moving object detection in the H.264/AVC compressed domain

This paper presents a moving object detection algorithm for H.264/AVC video streams that is applied in the compressed domain. The method is able to extract and analyze several syntax elements from any H.264/AVC-compliant bit stream. The number of analyzed syntax elements depends on the mode in which the method operates. The algorithm is able to perform either a spatiotemporal analysis in a single step or a two-step analysis that starts with a spatial analysis of each frame, followed by a temporal analysis of several subsequent frames. Thereby, in each mode either only (sub-)macroblock types and partition modes or, additionally, quantization parameters are analyzed. The evaluation of these syntax elements enables the algorithm to determine a “weight” for each 4×4 block of pixels that indicates the level of motion within this block. A final segmentation after creating these weights segments each frame to foreground and background and hence indicates the positions and sizes of all moving objects. Our experiments show that the algorithm is able to efficiently detect moving objects in the compressed domain and that it is configurable to process a large number of parallel bit streams in real time.

[1]  Steven Verstockt,et al.  Multi-view Object Localization in H.264/AVC Compressed Domain , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[2]  P. KaewTrakulPong,et al.  An Improved Adaptive Background Mixture Model for Real-time Tracking with Shadow Detection , 2002 .

[3]  Pascal Fua,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Multiple Object Tracking Using K-shortest Paths Optimization , 2022 .

[4]  André Kaup,et al.  A compressed domain change detection algorithm for RTP streams in video surveillance applications , 2011, 2011 IEEE 13th International Workshop on Multimedia Signal Processing.

[5]  Jan Nesvadba,et al.  Face detection in the compressed domain , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[6]  Henning Schulzrinne,et al.  RTP: A Transport Protocol for Real-Time Applications , 1996, RFC.

[7]  André Kaup,et al.  Compressed domain moving object detection by spatio-temporal analysis of H.264/AVC syntax elements , 2015, 2015 Picture Coding Symposium (PCS).

[8]  Zoran Zivkovic,et al.  Improved adaptive Gaussian mixture model for background subtraction , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[9]  Michael Schöberl,et al.  Change detection in JPEG 2000 compressed video , 2010 .

[10]  R. Venkatesh Babu,et al.  Fast moving-object detection in H.264/AVC compressed domain for video surveillance , 2013, 2013 Fourth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics (NCVPRIPG).

[11]  Peter Lambert,et al.  Moving object detection in the H.264/AVC compressed domain for video surveillance applications , 2009, J. Vis. Commun. Image Represent..

[12]  Ajay Luthra,et al.  Overview of the H.264/AVC video coding standard , 2003, IEEE Trans. Circuits Syst. Video Technol..

[13]  Athanassios N. Skodras,et al.  Moving object detection in the H.264 compressed domain , 2010, 2010 IEEE International Conference on Imaging Systems and Techniques.

[14]  Rik Van de Walle,et al.  Adaptive Background Subtraction in H.264/AVC Bitstreams based on Macroblock Sizes , 2011, VISAPP.

[15]  Fatih Murat Porikli,et al.  CDnet 2014: An Expanded Change Detection Benchmark Dataset , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[16]  Mark Goadrich,et al.  The relationship between Precision-Recall and ROC curves , 2006, ICML.

[17]  Henri Nicolas,et al.  Compressed domain aided analysis of traffic surveillance videos , 2009, 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC).

[18]  Munchurl Kim,et al.  Moving Object Detection and Tracking Using a Spatio-Temporal Graph in H.264/AVC Bitstreams for Video Surveillance , 2012, IEEE Transactions on Multimedia.

[19]  Satoshi Goto,et al.  Examination of a tracking and detection method using compressed domain information , 2013, 2013 Picture Coding Symposium (PCS).

[20]  Ivan V. Bajic,et al.  Video Object Tracking in the Compressed Domain Using Spatio-Temporal Markov Random Fields , 2013, IEEE Transactions on Image Processing.

[21]  André Kaup,et al.  Compressed Domain Moving Object Detection based on H.264/AVC Macroblock Types , 2013, VISAPP.

[22]  André Kaup,et al.  Hybrid Person Detection and Tracking in H.264/AVC Video Streams , 2015, VISAPP.

[23]  Chen Yaowu,et al.  A Novel Scene Change Detection Algorithm for H.264/AVC Bitstreams , 2008, 2008 IEEE Pacific-Asia Workshop on Computational Intelligence and Industrial Application.