ViBe: A Universal Background Subtraction Algorithm for Video Sequences

This paper presents a technique for motion detection that incorporates several innovative mechanisms. For example, our proposed technique stores, for each pixel, a set of values taken in the past at the same location or in the neighborhood. It then compares this set to the current pixel value in order to determine whether that pixel belongs to the background, and adapts the model by choosing randomly which values to substitute from the background model. This approach differs from those based upon the classical belief that the oldest values should be replaced first. Finally, when the pixel is found to be part of the background, its value is propagated into the background model of a neighboring pixel. We describe our method in full details (including pseudo-code and the parameter values used) and compare it to other background subtraction techniques. Efficiency figures show that our method outperforms recent and proven state-of-the-art methods in terms of both computation speed and detection rate. We also analyze the performance of a downscaled version of our algorithm to the absolute minimum of one comparison and one byte of memory per pixel. It appears that even such a simplified version of our algorithm performs better than mainstream techniques.

[1]  Bruno O. Shubert,et al.  Random variables and stochastic processes , 1979 .

[2]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[3]  John G. Proakis,et al.  Probability, random variables and stochastic processes , 1985, IEEE Trans. Acoust. Speech Signal Process..

[4]  Jitendra Malik,et al.  Robust Multiple Car Tracking with Occlusion Reasoning , 1994, ECCV.

[5]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[6]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[8]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[9]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[10]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[12]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[13]  Touradj Ebrahimi,et al.  Video object extraction based on adaptive background and statistical change detection , 2000, IS&T/SPIE Electronic Imaging.

[14]  Alan M. McIvor,et al.  Background Subtraction Techniques , 2000 .

[15]  Anil K. Jain,et al.  A background model initialization algorithm for video surveillance , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[16]  Anil K. Jain,et al.  A background model initialization algorithm for video surveillance , 2001, Proceedings Eighth IEEE International Conference on Computer Vision. ICCV 2001.

[17]  P. KaewTrakulPong,et al.  An Improved Adaptive Background Mixture Model for Real-time Tracking with Shadow Detection , 2002 .

[18]  L. Davis,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002, Proc. IEEE.

[19]  P. Wayne Power,et al.  Understanding Background Mixture Models for Foreground Segmentation , 2002 .

[20]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .

[21]  Qi Tian,et al.  Foreground object detection from videos containing complex background , 2003, MULTIMEDIA '03.

[22]  Kazuhiko Sumi,et al.  Background subtraction based on cooccurrence of image variations , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[23]  Rita Cucchiara,et al.  Detecting Moving Objects, Ghosts, and Shadows in Video Streams , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  Eero P. Simoncelli,et al.  On Advances in Statistical Modeling of Natural Images , 2004, Journal of Mathematical Imaging and Vision.

[25]  Z. Zivkovic Improved adaptive Gaussian mixture model for background subtraction , 2004, ICPR 2004.

[26]  Qi Tian,et al.  Statistical modeling of complex backgrounds for foreground object detection , 2004, IEEE Transactions on Image Processing.

[27]  Antoine Manzanera,et al.  A Robust and Computationally Efficient Motion Detection Algorithm Based on Sigma-Delta Background Estimation , 2004, ICVGIP.

[28]  Nikos Paragios,et al.  Motion-based background subtraction using adaptive kernel density estimation , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[29]  Massimo Piccardi,et al.  Background subtraction techniques: a review , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[30]  Larry S. Davis,et al.  Background modeling and subtraction by codebook construction , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[31]  Reinhard Klette,et al.  Robust background subtraction and maintenance , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[32]  James W. Davis,et al.  Robust Background-Subtraction for Person Detection in Thermal Imagery , 2004, 2004 Conference on Computer Vision and Pattern Recognition Workshop.

[33]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[34]  Soraia Raupp Musse,et al.  Background Subtraction and Shadow Detection in Grayscale Video Sequences , 2005, XVIII Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI'05).

[35]  Larry S. Davis,et al.  Real-time foreground-background segmentation using codebook model , 2005, Real Time Imaging.

[36]  Scott Cohen,et al.  Background estimation as a labeling problem , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[37]  Dar-Shyang Lee,et al.  Effective Gaussian mixture learning for video background subtraction , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Badrinath Roysam,et al.  Image change detection algorithms: a systematic survey , 2005, IEEE Transactions on Image Processing.

[39]  Yang Wang,et al.  A probabilistic approach for foreground and shadow segmentation in monocular image sequences , 2005, Pattern Recognit..

[40]  Helmut E. Bez,et al.  A practical adaptive approach for dynamic background subtraction using an invariant colour model and object tracking , 2005, Pattern Recognit. Lett..

[41]  Yaser Sheikh,et al.  Bayesian modeling of dynamic scenes for object detection , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Soraia Raupp Musse,et al.  A Background Subtraction Model Adapted to Illumination Changes , 2006, 2006 International Conference on Image Processing.

[43]  Marc Van Droogenbroeck,et al.  Robust Analysis of Silhouettes by Morphological Size Distributions , 2006, ACIVS.

[44]  James W. Davis,et al.  Background-Subtraction in Thermal Imagery Using Contour Saliency , 2007, International Journal of Computer Vision.

[45]  Real-time outdoor video surveillance with robust foreground extraction and object tracking via multi-state transition management , 2006 .

[46]  Jwu-Sheng Hu,et al.  Robust Background Subtraction with Shadow and Highlight Removal for Indoor Surveillance , 2006, IROS.

[47]  Yang Wang,et al.  A dynamic conditional random field model for foreground and shadow segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Ferdinand van der Heijden,et al.  Efficient adaptive density estimation per image pixel for the task of background subtraction , 2006, Pattern Recognit. Lett..

[49]  Jwu-Sheng Hu,et al.  Robust Background Subtraction with Shadow and Highlight Removal for Indoor Surveillance , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[50]  Bang Jun Lei,et al.  Real-time outdoor video surveillance with robust foreground extraction and object tracking via multi-state transition management , 2006, Pattern Recognit. Lett..

[51]  David Suter,et al.  Background Subtraction Based on a Robust Consensus Method , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[52]  Eduardo Monari,et al.  Fusion of background estimation approaches for motion detection in non-static backgrounds , 2007, 2007 IEEE Conference on Advanced Video and Signal Based Surveillance.

[53]  David Suter,et al.  A consensus-based method for tracking: Modelling background scenario and foreground appearance , 2007, Pattern Recognit..

[54]  Max Mignotte,et al.  Statistical background subtraction using spatial cues , 2007, IEEE Transactions on Circuits and Systems for Video Technology.

[55]  Antoine Manzanera,et al.  A new motion detection algorithm based on Sigma-Delta background estimation , 2007, Pattern Recognit. Lett..

[56]  Xavier Maldague,et al.  Outdoor infrared video surveillance: A novel dynamic technique for the subtraction of a changing background of IR images , 2007 .

[57]  Antoine Manzanera Sigma-Delta Background Subtraction and the Zipf Law , 2007, CIARP.

[58]  Robert G. Abbott,et al.  Multiple target tracking with lazy background subtraction and connected components analysis , 2009, Machine Vision and Applications.

[59]  Σ-Δ Background Subtraction and the Zipf Law , 2007 .

[60]  Mubarak Shah,et al.  Automatically Tuning Background Subtraction Parameters using Particle Swarm Optimization , 2007, 2007 IEEE International Conference on Multimedia and Expo.

[61]  Thomas S. Huang,et al.  Robust estimation of foreground in surveillance videos by sparse error estimation , 2008, 2008 19th International Conference on Pattern Recognition.

[62]  ヴァン・ドローヘンブルーク、マルク,et al.  Visual background extractor , 2008 .

[63]  Hélène Laurent,et al.  Review and evaluation of commonly-implemented background subtraction algorithms , 2008, 2008 19th International Conference on Pattern Recognition.

[64]  Monica N. Nicolescu,et al.  Non-parametric statistical background modeling for efficient foreground region detection , 2008, Machine Vision and Applications.

[65]  Sidney S. Fels,et al.  Evaluation of Background Subtraction Algorithms with Post-Processing , 2008, 2008 IEEE Fifth International Conference on Advanced Video and Signal Based Surveillance.

[66]  Lionel Lacassagne,et al.  High performance motion detection: some trends toward new embedded architectures for vision systems , 2008, Journal of Real-Time Image Processing.

[67]  Volkan Cevher,et al.  Compressive Sensing for Background Subtraction , 2008, ECCV.

[68]  Shireen Elhabian,et al.  Moving Object Detection in Spatial Domain using Background Removal Techniques - State-of-Art , 2008 .

[69]  Lucia Maddalena,et al.  A Self-Organizing Approach to Background Subtraction for Visual Surveillance Applications , 2008, IEEE Transactions on Image Processing.

[70]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..

[71]  Du-Ming Tsai,et al.  Independent Component Analysis-Based Background Subtraction for Indoor Surveillance , 2009, IEEE Transactions on Image Processing.

[72]  Cláudio Rosito Jung,et al.  Efficient Background Subtraction and Shadow Removal for Monochromatic Video Sequences , 2009, IEEE Transactions on Multimedia.

[73]  Lionel Lacassagne,et al.  Motion detection: Fast and robust algorithms for embedded systems , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[74]  Jen-Hui Chuang,et al.  Learning a Scene Background Model via Classification , 2009, IEEE Transactions on Signal Processing.

[75]  Marc Van Droogenbroeck,et al.  ViBE: A powerful random technique to estimate the background in video sequences , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[76]  D. Manjula,et al.  An Efficient Foreground Detection Algorithm for Visual Surveillance System , 2009 .

[77]  Nuno Vasconcelos,et al.  Spatiotemporal Saliency in Dynamic Scenes , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[78]  Chung-Cheng Chiu,et al.  A Robust Object Segmentation System Using a Probability-Based Background Extraction Algorithm , 2010, IEEE Transactions on Circuits and Systems for Video Technology.

[79]  Bertrand Vachon,et al.  Statistical Background Modeling for Foreground Detection: A Survey , 2010 .

[80]  Mingjun Wu,et al.  Spatio-temporal context for codebook-based dynamic background subtraction , 2010 .

[81]  Guillaume-Alexandre Bilodeau,et al.  A Multiscale Region-Based Motion Detection and Background Subtraction Algorithm , 2010, Sensors.