Metrics for Objective Evaluation of Background Subtraction Algorithms

Although a large number of background subtraction (BS) algorithms have been proposed, relevant objective metrics for evaluating these algorithms are still lacking. In this paper, empirical discrepancy metrics, which quantify the spatial accuracy and temporal stability of estimated masks by taking into account the potential inaccuracy of reference masks, the location of the pixel errors relative to the border of reference masks as well as the type of errors, are presented for evaluating the performance of BS algorithms. To validate the proposed metrics, they are applied to tune the optimal parameters of LBP-based background subtraction algorithm, and the experimental results confirm the efficiency of them.

[1]  Hélène Laurent,et al.  Review and evaluation of commonly-implemented background subtraction algorithms , 2008, 2008 19th International Conference on Pattern Recognition.

[2]  Marko Heikkilä,et al.  A texture-based method for modeling the background and detecting moving objects , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Paulo Villegas,et al.  Perceptually-weighted evaluation criteria for segmentation masks in video sequences , 2004, IEEE Transactions on Image Processing.

[4]  Horst Bischof,et al.  Performance evaluation metrics for motion detection and tracking , 2004, ICPR 2004.

[5]  I. Haritaoglu,et al.  Background and foreground modeling using nonparametric kernel density estimation for visual surveillance , 2002 .

[6]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[7]  Juan Rosell-Ortega,et al.  Background modelling in demanding situations with confidence measure , 2008, ICPR 2008.

[8]  R. A. Bailey,et al.  Design of comparative experiments , 2008 .

[9]  Touradj Ebrahimi,et al.  Objective evaluation of segmentation quality using spatio-temporal context , 2002, Proceedings. International Conference on Image Processing.

[10]  Nong Sang,et al.  Background subtraction using shape and colour information , 2010 .

[11]  Parvaneh Saeedi,et al.  Robust region-based background subtraction and shadow removing using color and gradient information , 2008, 2008 19th International Conference on Pattern Recognition.

[12]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).