Performance Evaluation of Surveillance Systems Under Varying Conditions

Effectively evaluating the performance of moving object detection and tracking algorithms is an important step towards attaining robust digital video surveillance systems with sufficient accuracy for practical applications. As systems become more complex and achieve greater robustness, the ability to quantitatively assess performance is needed in order to continuously improve performance. In this paper, we refine the methods used to estimate performance and use these methods to measure the performance of our system under several different conditions including: indoor/outdoor, different weather conditions (precipitation, wind, and brightness), different cameras/viewpoints, and as a standard benchmark, the PETS 2001 datasets. To test the extensibility/validity of our results, we have also evaluated our system on four longer data sets (20-30 min each) from four different cameras. We evaluate the performance of the background subtraction alone and with a simple tracking system using two different sets of metrics. Visualization of the performance results has proven critical for understanding the weaknesses of the