Dynamic background estimation and complementary learning for pixel-wise foreground/background segmentation

Change and motion detection plays a basic and guiding role in surveillance video analysis. Since most outdoor surveillance videos are taken in native and complex environments, these "static" backgrounds change in some unknown patterns, which make perfect foreground extraction very difficult. This paper presents two universal modifications for pixel-wise foreground/background segmentation: dynamic background estimation and complementary learning. These two modifications are embedded in three classic background subtraction algorithms: probability based background subtraction (Gaussian mixture model, GMM), sample based background subtraction (visual background extractor, ViBe) and code words based background subtraction (code book, CB). Experiments on several popular public datasets prove the effectiveness and real-time performance of the proposed method. Both GMM and CB with the proposed modifications have better performance than the original versions. Especially, ViBe with the modifications outperforms some state-of-art algorithms presented on the CHANGEDETECTION website. An efficient pixel-wise foreground/background segmentation algorithm.Complex dynamic background modeling by dynamic background estimation.Accurate foreground objects extraction by complementary learning.

[1]  L. Davis,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002, Proc. IEEE.

[2]  Paul L. Rosin,et al.  Evaluation of global image thresholding for change detection , 2003, Pattern Recognit. Lett..

[3]  Tieniu Tan,et al.  Foreground Object Detection Using Top-Down Information Based on EM Framework , 2012, IEEE Transactions on Image Processing.

[4]  Hélène Laurent,et al.  Comparative study of background subtraction algorithms , 2010, J. Electronic Imaging.

[5]  Marko Heikkilä,et al.  A texture-based method for modeling the background and detecting moving objects , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Richard Szeliski,et al.  A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Geoffrey H. Ball,et al.  ISODATA, A NOVEL METHOD OF DATA ANALYSIS AND PATTERN CLASSIFICATION , 1965 .

[8]  Pong C. Yuen,et al.  Object motion detection using information theoretic spatio-temporal saliency , 2009, Pattern Recognit..

[9]  Larry S. Davis,et al.  Background modeling and subtraction by codebook construction , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[10]  Marc Van Droogenbroeck,et al.  ViBe: A Universal Background Subtraction Algorithm for Video Sequences , 2011, IEEE Transactions on Image Processing.

[11]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  Takeo Kanade,et al.  Introduction to the Special Section on Video Surveillance , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Guillaume-Alexandre Bilodeau,et al.  SuBSENSE: A Universal Change Detection Method With Local Adaptive Sensitivity , 2015, IEEE Transactions on Image Processing.

[14]  Matti Pietikäinen,et al.  Modeling pixel process with scale invariant local patterns for background subtraction in complex scenes , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[15]  David Suter,et al.  A consensus-based method for tracking: Modelling background scenario and foreground appearance , 2007, Pattern Recognit..

[16]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .

[17]  Nuno Vasconcelos,et al.  Spatiotemporal Saliency in Dynamic Scenes , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Fatih Murat Porikli,et al.  Changedetection.net: A new change detection benchmark dataset , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[19]  Marc Van Droogenbroeck,et al.  Background subtraction: Experiments and improvements for ViBe , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[20]  Larry S. Davis,et al.  Real-time foreground-background segmentation using codebook model , 2005, Real Time Imaging.

[21]  Z. Zivkovic.  Improved adaptive Gaussian mixture model for background subtraction , 2004, ICPR 2004.

[22]  Mohan M. Trivedi,et al.  Analysis and detection of shadows in video streams: a comparative evaluation , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[23]  William T. Freeman,et al.  Comparison of graph cuts with belief propagation for stereo, using identical MRF parameters , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[24]  Nikos Komodakis,et al.  MRF Energy Minimization and Beyond via Dual Decomposition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  J.M. Ferryman,et al.  PETS Metrics: On-Line Performance Evaluation Service , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[26]  Zhenhua Guo,et al.  Background Subtraction with Dynamic Noise Sampling and Complementary Learning , 2014, 2014 22nd International Conference on Pattern Recognition.

[27]  Shengping Zhang,et al.  Dynamic background modeling and subtraction using spatio-temporal local binary patterns , 2008, 2008 15th IEEE International Conference on Image Processing.

[28]  Tao Xiang,et al.  Background Subtraction with Dirichlet Processes , 2012, ECCV.

[29]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Liang Lin,et al.  Integrating Graph Partitioning and Matching for Trajectory Analysis in Video Surveillance , 2012, IEEE Transactions on Image Processing.

[31]  Jian-Huang Lai,et al.  Complex Background Subtraction by Pursuing Dynamic Spatio-Temporal Models , 2014, IEEE Transactions on Image Processing.

[32]  Yaser Sheikh,et al.  Bayesian modeling of dynamic scenes for object detection , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Patrick Bouthemy,et al.  Simultaneous Motion Detection and Background Reconstruction with a Conditional Mixed-State Markov Random Field , 2011, International Journal of Computer Vision.

[34]  Fatih Porikli,et al.  Performance Evaluation of Object Detection and Tracking Systems , 2006 .

[35]  Chih-Hsien Hsia,et al.  Hierarchical Method for Foreground Detection Using Codebook Model , 2011, IEEE Trans. Circuits Syst. Video Technol..

[36]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[37]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[38]  Marc Van Droogenbroeck,et al.  ViBE: A powerful random technique to estimate the background in video sequences , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[39]  Stan Z. Li,et al.  Markov Random Field Modeling in Image Analysis , 2001, Computer Science Workbench.

[40]  Hai Jin,et al.  Integrating Spatio-Temporal Context With Multiview Representation for Object Recognition in Visual Surveillance , 2011, IEEE Transactions on Circuits and Systems for Video Technology.