A Consistent Metric for Performance Evaluation of Multi-Object Filters

The concept of a miss-distance, or error, between a reference quantity and its estimated/controlled value, plays a fundamental role in any filtering/control problem. Yet there is no satisfactory notion of a miss-distance in the well-established field of multi-object filtering. In this paper, we outline the inconsistencies of existing metrics in the context of multi-object miss-distances for performance evaluation. We then propose a new mathematically and intuitively consistent metric that addresses the drawbacks of current multi-object performance evaluation metrics.

[1]  Oliver E. Drummond,et al.  Performance metrics for multiple-sensor multiple-target tracking , 2000, SPIE Defense + Commercial Sensing.

[2]  Oliver E. Drummond,et al.  Ambiguities in evaluating performance of multiple target tracking algorithms , 1992, Defense, Security, and Sensing.

[3]  Björn Hammarberg,et al.  Parameter estimation of human nerve C-fibers using matched filtering and multiple hypothesis tracking , 2002, IEEE Transactions on Biomedical Engineering.

[4]  Umberto Spagnolini,et al.  Multitarget detection/tracking for monostatic ground penetrating radar: application to pavement profiling , 1999, IEEE Trans. Geosci. Remote. Sens..

[5]  Ba-Ngu Vo,et al.  Analytic Implementations of the Cardinalized Probability Hypothesis Density Filter , 2007, IEEE Transactions on Signal Processing.

[6]  Peter Breeze,et al.  Point Processes and Their Statistical Inference , 1991 .

[7]  Samuel S. Blackman,et al.  Multiple-Target Tracking with Radar Applications , 1986 .

[8]  Ronald P. S. Mahler,et al.  Multitarget miss distance via optimal assignment , 2004, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[9]  D. M. Kocak,et al.  Computer vision techniques for quantifying, tracking, and identifying bioluminescent plankton , 1999 .

[10]  Ronald P. S. Mahler,et al.  Statistical Multisource-Multitarget Information Fusion , 2007 .

[11]  G. Matheron Random Sets and Integral Geometry , 1976 .

[12]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[13]  Ba-Ngu Vo,et al.  The Gaussian Mixture Probability Hypothesis Density Filter , 2006, IEEE Transactions on Signal Processing.

[14]  A. Baddeley Errors in binary images and an $Lsp p$ version of the Hausdorff metric , 1992 .

[15]  David M. Lane,et al.  Robust tracking of multiple objects in sector-scan sonar image sequences using optical flow motion estimation , 1998 .

[16]  Aihua Xia,et al.  A new metric between distributions of point processes , 2007, Advances in Applied Probability.

[17]  Oliver E. Drummond Methodologies for performance evaluation of multitarget multisensor tracking , 1999, Optics & Photonics.

[18]  Esko Valkeila,et al.  An Introduction to the Theory of Point Processes, Volume II: General Theory and Structure, 2nd Edition by Daryl J. Daley, David Vere‐Jones , 2008 .

[19]  A. Karr,et al.  Point Processes and Their Statistical Inference. 2nd edn. , 1993 .

[20]  Ingemar J. Cox,et al.  An Efficient Implementation of Reid's Multiple Hypothesis Tracking Algorithm and Its Evaluation for the Purpose of Visual Tracking , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Daryl J. Daley,et al.  An Introduction to the Theory of Point Processes , 2013 .

[22]  Y. Bar-Shalom Tracking and data association , 1988 .

[23]  Hugh F. Durrant-Whyte,et al.  Simultaneous localization and mapping: part I , 2006, IEEE Robotics & Automation Magazine.

[24]  Hugh Durrant-Whyte,et al.  Simultaneous Localisation and Mapping ( SLAM ) : Part I The Essential Algorithms , 2006 .