Change detection by probabilistic segmentation from monocular view

We present a method for foreground/background video segmentation (change detection) in real-time that can be used, in applications such as background subtraction or analysis of surveillance cameras. Our approach implements a probabilistic segmentation based on the Quadratic Markov Measure Field models. This framework regularizes the likelihood of each pixel belonging to each one of the classes (background or foreground). We propose a new likelihood that takes into account two cases: the first one is when the background is static and the foreground might be static or moving (Static Background Subtraction), the second one is when the background is unstable and the foreground is moving (Unstable Background Subtraction). Moreover, our likelihood is robust to illumination changes, cast shadows and camouflage situations. We implement a parallel version of our algorithm in CUDA using a NVIDIA Graphics Processing Unit in order to fulfill real-time execution requirements.

[1]  Wen-mei W. Hwu,et al.  Programming Massively Parallel Processors with CUDA (audio course) , 2011 .

[2]  Andrew Blake,et al.  Probabilistic Fusion of Stereo with Color and Contrast for Bilayer Segmentation , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[4]  Gerhard Rigoll,et al.  Background segmentation with feedback: The Pixel-Based Adaptive Segmenter , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[5]  Mariano Rivera,et al.  Variational Viewpoint of the Quadratic Markov Measure Field Models: Theory and Algorithms , 2012, IEEE Transactions on Image Processing.

[6]  Mariano Rivera,et al.  Quadratic Markovian Probability Fields for Image Binary Segmentation , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[7]  Sang Uk Lee,et al.  Robust bilayer video segmentation by adaptive propagation of global shape and local appearance , 2010, J. Vis. Commun. Image Represent..

[8]  Osama Masoud,et al.  Moving Shadow Detection with Low- and Mid-Level Reasoning , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[9]  Mariano Rivera,et al.  Entropy-Controlled Quadratic Markov Measure Field Models for Efficient Image Segmentation , 2007, IEEE Transactions on Image Processing.

[10]  Harry Shum,et al.  Background Cut , 2006, ECCV.

[11]  A. Murat Tekalp,et al.  4.10 – Video Segmentation , 2005 .

[12]  Luc Van Gool,et al.  GPU-Based Foreground-Background Segmentation using an Extended Colinearity Criterion , 2005 .

[13]  Simone Calderara,et al.  HECOL: Homography and epipolar-based consistent labeling for outdoor park surveillance , 2008, Comput. Vis. Image Underst..

[14]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[15]  Andrew Blake,et al.  Bi-layer segmentation of binocular stereo video , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[16]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .

[17]  Andrew Blake,et al.  Probabilistic Fusion of Stereo with Color and Contrast for Bi-Layer Segmentation , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[19]  Rainer Stiefelhagen,et al.  Improving foreground segmentations with probabilistic superpixel Markov random fields , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[20]  Fatih Murat Porikli,et al.  Changedetection.net: A new change detection benchmark dataset , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[21]  Thierry Bouwmans,et al.  Background subtraction via incremental maximum margin criterion: a discriminative subspace approach , 2012, Machine Vision and Applications.

[22]  Mariano Rivera,et al.  Bayesian segmentation of range images of polyhedral objects using entropy-controlled quadratic Markov measure field models. , 2008, Applied optics.

[23]  Yaser Sheikh,et al.  Bayesian modeling of dynamic scenes for object detection , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[24]  Mariano Rivera,et al.  Binary Segmentation of Video Sequences in Real Time , 2010, 2010 Ninth Mexican International Conference on Artificial Intelligence.

[25]  Marc Van Droogenbroeck,et al.  ViBe: A Universal Background Subtraction Algorithm for Video Sequences , 2011, IEEE Transactions on Image Processing.

[26]  Atsushi Shimada,et al.  Evaluation report of integrated background modeling based on spatio-temporal features , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[27]  W. Eric L. Grimson,et al.  Learning Patterns of Activity Using Real-Time Tracking , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[28]  Ferdinand van der Heijden,et al.  Efficient adaptive density estimation per image pixel for the task of background subtraction , 2006, Pattern Recognit. Lett..

[29]  Wen-mei W. Hwu,et al.  GPU Computing Gems Jade Edition , 2011 .

[30]  Lucia Maddalena,et al.  The SOBS algorithm: What are the limits? , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[31]  Yuji Iwahori,et al.  GPU based extraction of moving objects without shadows under intensity changes , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[32]  RiveraMariano,et al.  Change detection by probabilistic segmentation from monocular view , 2014 .

[33]  Tiziana D'Orazio,et al.  Moving object segmentation by background subtraction and temporal analysis , 2006, Image Vis. Comput..

[34]  David Suter,et al.  A consensus-based method for tracking: Modelling background scenario and foreground appearance , 2007, Pattern Recognit..

[35]  Javier-Flavio Vigueras,et al.  Registration and interactive planar segmentation for stereo images of polyhedral scenes , 2010, Pattern Recognit..

[36]  Peter Carr,et al.  GPU Accelerated Multimodal Background Subtraction , 2008, 2008 Digital Image Computing: Techniques and Applications.

[37]  Tamás Szirányi,et al.  Bayesian Foreground and Shadow Detection in Uncertain Frame Rate Surveillance Videos , 2008, IEEE Transactions on Image Processing.

[38]  Xiang Pan,et al.  GSM-MRF based classification approach for real-time moving object detection , 2008 .

[39]  Lucia Maddalena,et al.  A Self-Organizing Approach to Background Subtraction for Visual Surveillance Applications , 2008, IEEE Transactions on Image Processing.

[40]  Richard Szeliski,et al.  A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[41]  Jorge P. Batista,et al.  Robust Segmentation Process to Detect Incidents on Highways , 2008, ICIAR.

[42]  Marc Van Droogenbroeck,et al.  Background subtraction: Experiments and improvements for ViBe , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[43]  Minglun Gong,et al.  Real-time foreground segmentation on GPUs using local online learning and global graph cut optimization , 2008, 2008 19th International Conference on Pattern Recognition.

[44]  Grantham K. H. Pang,et al.  Effective moving cast shadow detection for monocular color traffic image sequences , 2002 .

[45]  Tieniu Tan,et al.  An Illumination Invariant Change Detection Algorithm , 2002 .

[46]  Rubén Heras Evangelio,et al.  Complementary background models for the detection of static and moving objects in crowded environments , 2011, 2011 8th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[47]  Rubén Heras Evangelio,et al.  Static Object Detection Based on a Dual Background Model and a Finite-State Machine , 2011, EURASIP J. Image Video Process..

[48]  Touradj Ebrahimi,et al.  Shadow identification and classification using invariant color models , 2001, 2001 IEEE International Conference on Acoustics, Speech, and Signal Processing. Proceedings (Cat. No.01CH37221).

[49]  Amir Averbuch,et al.  A region-based MRF model for unsupervised segmentation of moving objects in image sequences , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[50]  Zhenjiang Miao,et al.  Foreground prediction for bilayer segmentation of videos , 2011, Pattern Recognit. Lett..

[51]  Irfan A. Essa,et al.  Bilayer Segmentation of Webcam Videos Using Tree-Based Classifiers , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Andrew Blake,et al.  "GrabCut" , 2004, ACM Trans. Graph..