A metric for sets of trajectories that is practical and mathematically consistent

Metrics on the space of sets of trajectories are important for scientists in the field of computer vision, machine learning, robotics, and general artificial intelligence. However, existing notions of closeness between sets of trajectories are either mathematically inconsistent or of limited practical use. In this paper, we outline the limitations in the current mathematically-consistent metrics, which are based on OSPA (Schuhmacher et al. 2008); and the inconsistencies in the heuristic notions of closeness used in practice, whose main ideas are common to the CLEAR MOT measures (Keni and Rainer 2008) widely used in computer vision. In two steps, we then propose a new intuitive metric between sets of trajectories and address these limitations. First, we explain a solution that leads to a metric that is hard to compute. Then we modify this formulation to obtain a metric that is easy to compute while keeping the useful properties of the previous metric. Our notion of closeness is the first demonstrating the following three features: the metric 1) can be quickly computed, 2) incorporates confusion of trajectories' identity in an optimal way, and 3) is a metric in the mathematical sense.

[1]  Oliver E. Drummond,et al.  Ambiguities in evaluating performance of multiple target tracking algorithms , 1992, Defense, Security, and Sensing.

[2]  L. G. H. Cijan.  A polynomial algorithm in linear programming , 1979 .

[3]  Ramakant Nevatia,et al.  Multi-target tracking by online learning of non-linear motion patterns and robust appearance models , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  L. Khachiyan.  Polynomial algorithms in linear programming , 1980 .

[5]  Dimitrios Makris,et al.  Performance evaluation of object tracking algorithms , 2007 .

[6]  Oliver E. Drummond.  Methodologies for performance evaluation of multitarget multisensor tracking , 1999, Optics & Photonics.

[7]  Fatih Porikli,et al.  Performance Evaluation of Object Detection and Tracking Systems , 2006 .

[8]  F. Porikli.  Trajectory Distance Metric Using Hidden Markov Model Based Representation , 2004 .

[9]  Stephen P. Boyd,et al.  Distributed Optimization and Statistical Learning via the Alternating Direction Method of Multipliers , 2011, Found. Trends Mach. Learn..

[10]  Mohammad Rostami,et al.  Testing Fine-Grained Parallelism for the ADMM on a Factor-Graph , 2016, 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW).

[11]  Rastko R. Selmic,et al.  Measuring Distance between Unordered Sets of Different Sizes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  M. Kendall.  A NEW MEASURE OF RANK CORRELATION , 1938 .

[13]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[14]  Oliver E. Drummond,et al.  Performance metrics for multiple-sensor multiple-target tracking , 2000, SPIE Defense + Commercial Sensing.

[15]  Margrit Betke,et al.  Tracking a large number of objects from multiple views , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[16]  Ronald P. S. Mahler,et al.  Multitarget miss distance via optimal assignment , 2004, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[17]  Peter N. Yianilos,et al.  Data structures and algorithms for nearest neighbor search in general metric spaces , 1993, SODA '93.

[18]  Oliver E. Drummond,et al.  Performance evaluation methods for multiple-target-tracking algorithms , 1991, Defense, Security, and Sensing.

[19]  Elena Deza,et al.  Encyclopedia of Distances , 2014 .

[20]  Margrit Betke,et al.  Tracking Large Variable Numbers of Objects in Clutter , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Rainer Stiefelhagen,et al.  Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics , 2008, EURASIP J. Image Video Process..

[22]  A. Cayley,et al.  LXXVII. Note on the theory of permutations , 1849 .

[23]  Konrad Schindler,et al.  Challenges of Ground Truth Evaluation of Multi-target Tracking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[24]  Ian D. Reid,et al.  Stable multi-target tracking in real-time surveillance video , 2011, CVPR 2011.

[25]  Edward K. Kao,et al.  An information theoretic approach for tracker performance evaluation , 2009, ICCV.

[26]  Thia Kirubarajan,et al.  Estimation with Applications to Tracking and Navigation: Theory, Algorithms and Software , 2001 .

[27]  Branko Ristic,et al.  A Metric for Performance Evaluation of Multi-Target Tracking Algorithms , 2011, IEEE Transactions on Signal Processing.

[28]  Tim Ellis.  Performance metrics and methods for tracking in surveillance , 2002 .

[29]  Xiaofan He,et al.  A Track Quality Based Metric for Evaluating Performance of Multitarget Filters , 2013, IEEE Transactions on Aerospace and Electronic Systems.

[30]  Samuel S. Blackman,et al.  Multiple-Target Tracking with Radar Applications , 1986 .

[31]  Ian D. Reid,et al.  Guiding Visual Surveillance by Tracking Human Attention , 2009, BMVC.

[32]  Samuel J. Davey,et al.  Performance Assessment of Tracking Systems , 1996, Fourth International Symposium on Signal Processing and Its Applications.

[33]  Ba-Ngu Vo,et al.  A Consistent Metric for Performance Evaluation of Multi-Object Filters , 2008, IEEE Transactions on Signal Processing.

[34]  Osamu Fujita,et al.  Metrics based on average distance between sets , 2011, Japan Journal of Industrial and Applied Mathematics.

[35]  Thia Kirubarajan,et al.  Performance measures for multiple target tracking problems , 2011, 14th International Conference on Information Fusion.

[36]  Jakub Segen,et al.  Performance evaluation of people tracking systems , 1996, Proceedings Third IEEE Workshop on Applications of Computer Vision. WACV'96.

[37]  Rob J. Evans,et al.  A new performance metric for multiple target tracking based on optimal subpattern assignment , 2014, 17th International Conference on Information Fusion (FUSION).

[38]  James C. French,et al.  Clustering large datasets in arbitrary metric spaces , 1999, Proceedings 15th International Conference on Data Engineering (Cat. No.99CB36337).

[39]  Andrea Cavallaro,et al.  Tracking Multiple High-Density Homogeneous Targets , 2015, IEEE Transactions on Circuits and Systems for Video Technology.

[40]  Guilherme França,et al.  An explicit rate bound for over-relaxed ADMM , 2015, 2016 IEEE International Symposium on Information Theory (ISIT).

[41]  Daniel E. Clark,et al.  Incorporating track uncertainty into the OSPA metric , 2011, 14th International Conference on Information Fusion.

[42]  Margrit Betke,et al.  Thermal Imaging Reveals Significantly Smaller Brazilian Free-Tailed Bat Colonies Than Previously Estimated , 2008, Journal of Mammalogy.

[43]  Éva Tardos,et al.  Approximation algorithms for classification problems with pairwise relationships: metric labeling and Markov random fields , 2002, JACM.