A novel performance evaluation paradigm for automated video surveillance systems

Most existing performance evaluation methods concentrate on defining various metrics over a wide range of conditions and generating standard benchmarking video sequences to examine the effectiveness of a video tracking system. It is a common practice to incorporate a robustness margin or factor into the system/algorithm design. However, these methods, deterministic approaches, often lead to overdesign, thus increasing costs, or underdesign, causing frequent system failures. In order to overcome the aforementioned limitations, we propose an alternative framework to analyze the physics of the failure process via the concept of reliability. In comparison with existing approaches where system performance is evaluated based on a given benchmarking sequence, the advantage of our proposed framework lies in that a unified and statistical index is used to evaluate the performance of an automated video surveillance system independent of input sequences. Meanwhile, based on our proposed framework, the uncertainty problem of a failure process caused by the system’s complexity, imprecise measurements of the relevant physical constants and variables, and the indeterminate nature of future events can be addressed accordingly.