PETS Metrics: On-Line Performance Evaluation Service

This paper presents the PETS Metrics On-line Evaluation Service for computational visual surveillance algorithms. The service allows researchers to submit their algorithm results for evaluation against a set of applicable metrics. The results of the evaluation processes are publicly displayed allowing researchers to instantly view how their algorithm performs against previously submitted algorithms. The approach has been validated using seven motion segmentation algorithms.

[1]  Gary Bradski,et al.  Computer Vision Face Tracking For Use in a Perceptual User Interface , 1998 .

[2]  Larry S. Davis,et al.  A Robust Background Subtraction and Shadow Detection , 1999 .

[3]  James E. Black,et al.  A novel method for video tracking performance evaluation , 2003 .

[4]  A. Senior,et al.  Performance Evaluation of Surveillance Systems Under Varying Conditions , 2004 .

[5]  Azriel Rosenfeld,et al.  Detection and location of people in video images using adaptive fusion of color and edge information , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[6]  Touradj Ebrahimi,et al.  Objective evaluation of segmentation quality using spatio-temporal context , 2002, Proceedings. International Conference on Image Processing.

[7]  Bülent Sankur,et al.  Performance evaluation metrics for object-based video segmentation , 2000, 2000 10th European Signal Processing Conference.

[8]  W. Eric L. Grimson,et al.  Adaptive background mixture models for real-time tracking , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[9]  Larry S. Davis,et al.  Non-parametric Model for Background Subtraction , 2000, ECCV.

[10]  Robert T. Collins,et al.  An Open Source Tracking Testbed and Evaluation Web Site , 2005 .

[11]  D. Thirde,et al.  Evaluation of Motion Segmentation Quality for Aircraft Activity Surveillance , 2005, 2005 IEEE International Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance.

[12]  Kentaro Toyama,et al.  Wallflower: principles and practice of background maintenance , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[13]  Horst Bischof,et al.  Quantitative Evaluations of Motion Detection Algorithms for Surveillance Applications , 2003 .

[14]  Tim Ellis.  Performance metrics and methods for tracking in surveillance , 2002 .

[15]  Alex Pentland,et al.  Pfinder: Real-Time Tracking of the Human Body , 1997, IEEE Trans. Pattern Anal. Mach. Intell..