A Background Modeling and Foreground Detection Algorithm Using Scaling Coefficients Defined With a Color Model Called Lightness-Red-Green-Blue

This paper presents an algorithm for background modeling and foreground detection that uses scaling coefficients, which are defined with a new color model called lightness-red-green-blue (LRGB). They are employed to compare two images by finding pixels with scaled lightness. Three backgrounds are used: 1) verified background with pixels that are considered as background; 2) testing background with pixels that are tested several times to check if they belong to the background; and 3) final background that is a combination of the testing and verified background (the testing background is used in places, where the verified background is not defined). If a testing background pixel matches pixels from previous frames (the match is tested using scaling coefficients), it is copied to the verified background, otherwise the pixel is set as the weighted average of the corresponding pixels of the last input images. After the background is computed, foreground objects are detected by using the scaling coefficients and additional criteria. The algorithm was evaluated using the SABS data set, Wallflower data set and a subset of the CDnet 2014 data set. The average F measure and sensitivity with the SABS Data set were 0.7109 and 0.8725, respectively. In the Wallflower data set, the total number of errors was 5280 and the total F-measure was 0.9089. In the CDnet 2014 data set, the F-measure for the baseline test case was 0.8887 and for the shadow test case was 0.8300.

[1]  A R Smith,et al.  Color Gamut Transformation Pairs , 1978 .

[2]  Lucia Maddalena,et al.  A Self-Organizing Approach to Background Subtraction for Visual Surveillance Applications , 2008, IEEE Transactions on Image Processing.

[3]  Thierry Bouwmans,et al.  Background Modeling using Mixture of Gaussians for Foreground Detection - A Survey , 2008 .

[4]  Alexander Wong,et al.  PIRM: Fast background subtraction under sudden, local illumination changes via probabilistic illumination range modelling , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[5]  Sridha Sridharan,et al.  Real-Time Adaptive Foreground/Background Segmentation , 2005, EURASIP J. Adv. Signal Process..

[6]  Shireen Elhabian,et al.  Moving Object Detection in Spatial Domain using Background Removal Techniques - State-of-Art , 2008 .

[7]  Richard G. Baraniuk,et al.  Compressive Sensing , 2008, Computer Vision, A Reference Guide.

[8]  Changzhi Lv,et al.  An updating method of self-adaptive background for moving objects detection in video , 2008, International Conferences on Audio, Language and Image Processing.

[9]  Massimo Piccardi,et al.  Background subtraction techniques: a review , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[10]  Serhat Selcuk Bucak,et al.  Incremental Non-negative Matrix Factorization for Dynamic Background Modelling , 2007, PRIS.

[11]  Antoine Vacavant,et al.  A comprehensive review of background subtraction algorithms evaluated with synthetic and real videos , 2014, Comput. Vis. Image Underst..

[12]  Larry S. Davis,et al.  Background modeling and subtraction by codebook construction , 2004, 2004 International Conference on Image Processing, 2004. ICIP '04..

[13]  Xiaofei Wang,et al.  Adaptive Low Rank Approximation for Tensors , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[14]  Andrzej Cichocki,et al.  Total Variation Regularized Tensor RPCA for Background Subtraction From Compressive Measurements , 2015, IEEE Transactions on Image Processing.

[15]  Xiaochun Cao,et al.  Robust Foreground Detection Using Smoothness and Arbitrariness Constraints , 2014, ECCV.

[16]  Marc Van Droogenbroeck,et al.  ViBE: A powerful random technique to estimate the background in video sequences , 2009, 2009 IEEE International Conference on Acoustics, Speech and Signal Processing.

[17]  Jen-Hui Chuang,et al.  A probabilistic SVM approach for background scene initialization , 2002, Proceedings. International Conference on Image Processing.

[18]  Dongmei Chen,et al.  Change detection from remotely sensed images: From pixel-based to object-based approaches , 2013 .

[19]  Til Aach,et al.  Illumination-invariant change detection , 2000, 4th IEEE Southwest Symposium on Image Analysis and Interpretation.

[20]  Ezequiel López-Rubio,et al.  Local color transformation analysis for sudden illumination change detection , 2015, Image Vis. Comput..

[21]  Azriel Rosenfeld,et al.  Tracking Groups of People , 2000, Comput. Vis. Image Underst..

[22]  Thierry Bouwmans,et al.  Subspace Learning for Background Modeling: A Survey , 2009 .

[23]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[24]  G..  A Theory for Multiresolution Signal Decomposition : The Wavelet Representation , 2004 .

[25]  Yi Ma,et al.  Robust principal component analysis? , 2009, JACM.

[26]  Q. M. Jonathan Wu,et al.  Real-time Video Segmentation using Student's t Mixture Model , 2012 .

[27]  Shih-Chia Huang,et al.  A background model re-initialization method based on sudden luminance change detection , 2015, Eng. Appl. Artif. Intell..

[28]  Gunter Menz,et al.  Recent Advances in Remote Sensing Change Detection – A Review , 2014 .

[29]  David Suter,et al.  A re-evaluation of mixture of Gaussian background modeling [video signal processing applications] , 2005, Proceedings. (ICASSP '05). IEEE International Conference on Acoustics, Speech, and Signal Processing, 2005..

[30]  Volkan Cevher,et al.  Compressive Sensing for Background Subtraction , 2008, ECCV.

[31]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[32]  Mircea Nicolescu,et al.  A Novelty Detection Approach for Foreground Region Detection in Videos with Quasi-stationary Backgrounds , 2006, ISVC.

[33]  Konrad Schindler,et al.  Smooth Foreground-Background Segmentation for Video Processing , 2006, ACCV.

[34]  George Bebis,et al.  Robust Video-Based Surveillance by Integrating Target Detection with Tracking , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[35]  Edward J. Delp,et al.  Foreground segmentation with sudden illumination changes using a shading model and a Gaussianity test , 2011, 2011 7th International Symposium on Image and Signal Processing and Analysis (ISPA).

[36]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[37]  Lene Juul Pedersen,et al.  Illumination and Reflectance Estimation with its Application in Foreground Detection , 2015, Sensors.

[38]  Thierry Bouwmans,et al.  Recent Advanced Statistical Background Modeling for Foreground Detection - A Systematic Survey , 2011 .

[39]  Jesús Darío Romero Ávila.  An indoor fall detection system for patients and elders , 2015 .

[40]  José Carlos Príncipe,et al.  Adaptive background estimation using an information theoretic cost for hidden state estimation , 2011, The 2011 International Joint Conference on Neural Networks.

[41]  Shuicheng Yan,et al.  Online Robust PCA via Stochastic Optimization , 2013, NIPS.

[42]  Michael J. Black,et al.  Secrets of optical flow estimation and their principles , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[43]  R.G. Baraniuk,et al.  Compressive Sensing [Lecture Notes] , 2007, IEEE Signal Processing Magazine.

[44]  Alex Pentland,et al.  Pfinder: real-time tracking of the human body , 1996, Proceedings of the Second International Conference on Automatic Face and Gesture Recognition.

[45]  Bob Zhang,et al.  Background modeling methods in video analysis: A review and comparative evaluation , 2016, CAAI Trans. Intell. Technol..

[46]  Nizar Bouguila,et al.  Online variational learning of finite Dirichlet mixture models , 2012, Evol. Syst..

[47]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[48]  Stéphane Mallat,et al.  A Theory for Multiresolution Signal Decomposition: The Wavelet Representation , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[49]  Zoran Zivkovic,et al.  Improved adaptive Gaussian mixture model for background subtraction , 2004, Proceedings of the 17th International Conference on Pattern Recognition, 2004. ICPR 2004..

[50]  Gerhard Rigoll,et al.  Background segmentation with feedback: The Pixel-Based Adaptive Segmenter , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[51]  Xi Chen,et al.  Direct Robust Matrix Factorizatoin for Anomaly Detection , 2011, 2011 IEEE 11th International Conference on Data Mining.

[52]  Larry S. Davis,et al.  W4: Real-Time Surveillance of People and Their Activities , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[53]  Nicholas J. Tate,et al.  A critical synthesis of remotely sensed optical image change detection techniques , 2015 .

[54]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[55]  Chung-Hao Chen,et al.  Spatially adaptive illumination modeling for background subtraction , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[56]  Alvy Ray Smith,et al.  Color gamut transform pairs , 1978, SIGGRAPH.

[57]  Vittorio Murino,et al.  Background Subtraction for Automated Multisensor Surveillance: A Comprehensive Review , 2010, EURASIP J. Adv. Signal Process..

[58]  M.M. Trivedi,et al.  Vision modules for a multi-sensory bridge monitoring approach , 2004, Proceedings. The 7th International IEEE Conference on Intelligent Transportation Systems (IEEE Cat. No.04TH8749).

[59]  Yasushi Yagi,et al.  Efficient Background Subtraction under Abrupt Illumination Variations , 2012, ACCV.

[60]  Fatih Murat Porikli,et al.  CDnet 2014: An Expanded Change Detection Benchmark Dataset , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[61]  Federico Tombari,et al.  Robust and accurate change detection under sudden illumination variations , 2007 .

[62]  Namrata Vaswani,et al.  Real-time Robust Principal Components' Pursuit , 2010, 2010 48th Annual Allerton Conference on Communication, Control, and Computing (Allerton).

[63]  Enrique Mérida Casermeiro,et al.  Video Object Segmentation with Multivalued Neural Networks , 2008, 2008 Eighth International Conference on Hybrid Intelligent Systems.

[64]  Seong-Dae Kim,et al.  Robust change detection by global-illumination- change compensation and noise-adaptive thresholding , 2004 .

[65]  D. Scott Wills,et al.  Chromatic Sensitivity of Illumination Change Compensation Techniques , 2010, ISVC.

[66]  Borko Furht,et al.  Neural Network Approach to Background Modeling for Video Object Segmentation , 2007, IEEE Transactions on Neural Networks.

[67]  Federico Tombari,et al.  A robust measure for visual correspondence , 2007, 14th International Conference on Image Analysis and Processing (ICIAP 2007).

[68]  Badrinath Roysam,et al.  Image change detection algorithms: a systematic survey , 2005, IEEE Transactions on Image Processing.

[69]  A. Oppenheim,et al.  Nonlinear filtering of multiplied and convolved signals , 1968 .

[70]  Benjamin Höferlin,et al.  Evaluation of background subtraction techniques for video surveillance , 2011, CVPR 2011.

[71]  Xiaodong Li,et al.  Stable Principal Component Pursuit , 2010, 2010 IEEE International Symposium on Information Theory.

[72]  Til Aach,et al.  Bayesian algorithms for adaptive change detection in image sequences using Markov random fields , 1995, Signal Process. Image Commun..

[73]  Thierry Bouwmans,et al.  Traditional and recent approaches in background modeling for foreground detection: An overview , 2014, Comput. Sci. Rev..

[74]  Ashbindu Singh,et al.  Review Article Digital change detection techniques using remotely-sensed data , 1989 .

[75]  David Suter,et al.  Background Subtraction Based on a Robust Consensus Method , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[76]  D. Titterington.  Recursive Parameter Estimation Using Incomplete Data , 1984 .

[77]  Lucia Maddalena,et al.  The SOBS algorithm: What are the limits? , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[78]  Soon Ki Jung,et al.  OR-PCA with MRF for Robust Foreground Detection in Highly Dynamic Backgrounds , 2014, ACCV.

[79]  ANTONIN CHAMBOLLE,et al.  An Algorithm for Total Variation Minimization and Applications , 2004, Journal of Mathematical Imaging and Vision.

[80]  Guillaume-Alexandre Bilodeau,et al.  SuBSENSE: A Universal Change Detection Method With Local Adaptive Sensitivity , 2015, IEEE Transactions on Image Processing.

[81]  Q. M. Jonathan Wu,et al.  Real-timeVideoSegmentation Using Student'stMixture Model , 2012, ANT/MobiWIS.

[82]  Soon Ki Jung,et al.  Decomposition into Low-rank plus Additive Matrices for Background/Foreground Separation: A Review for a Comparative Evaluation with a Large-Scale Dataset , 2015, Comput. Sci. Rev..

[83]  Christopher R. Wren,et al.  Waviz : Spectral Similarity for Object Detection , 2004 .

[84]  Takashi Toriu,et al.  Making Background Subtraction Robust to Various Illumination Changes , 2011 .

[85]  Donghui Wang,et al.  Background subtraction based on nonparametric Bayesian estimation , 2011, International Conference on Digital Image Processing.

[86]  Allen R. Hanson,et al.  Background modeling using adaptive pixelwise kernel variances in a hybrid feature space , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[87]  Thierry Bouwmans,et al.  Background Subtraction for Visual Surveillance , 2012, Handbook of Soft Computing for Video Surveillance.

[88]  Til Aach,et al.  Illumination-Invariant Change Detection Using a Statistical Colinearity Criterion , 2001, DAGM-Symposium.

[89]  Ferdinand van der Heijden,et al.  Efficient adaptive density estimation per image pixel for the task of background subtraction , 2006, Pattern Recognit. Lett..

[90]  Til Aach,et al.  Bayesian spatio-temporal motion detection under varying illumination , 2000, 2000 10th European Signal Processing Conference.

[91]  Donald P. Greenberg,et al.  Color spaces for computer graphics , 1978, SIGGRAPH.

[92]  Qi Tian,et al.  Foreground object detection from videos containing complex background , 2003, MULTIMEDIA '03.

[93]  Hélène Laurent,et al.  Review and evaluation of commonly-implemented background subtraction algorithms , 2008, 2008 19th International Conference on Pattern Recognition.

[94]  Gonzalo Pajares,et al.  Performance Analysis of Homomorphic Systems for Image Change Detection , 2005, IbPRIA.

[95]  D. Lu,et al.  Change detection techniques , 2004 .

[96]  Loong Fah Cheong,et al.  Block-Sparse RPCA for Consistent Foreground Detection , 2012, ECCV.

[97]  Thierry Bouwmans,et al.  Foreground Detection via Robust Low Rank Matrix Decomposition Including Spatio-Temporal Constraint , 2012, ACCV Workshops.

[98]  Nigel J. B. McFarlane,et al.  Segmentation and tracking of piglets in images , 1995, Machine Vision and Applications.

[99]  Thierry Bouwmans,et al.  Robust PCA via Principal Component Pursuit: A review for a comparative evaluation in video surveillance , 2014, Comput. Vis. Image Underst..