Performance Measures and a Data Set for Multi-target, Multi-camera Tracking

To help accelerate progress in multi-target, multi-camera tracking systems, we present (i) a new pair of precision-recall measures of performance that treats errors of all types uniformly and emphasizes correct identification over sources of error; (ii) the largest fully-annotated and calibrated data set to date with more than 2 million frames of 1080 p, 60 fps video taken by 8 cameras observing more than 2,700 identities over 85 min; and (iii) a reference software system as a comparison baseline. We show that (i) our measures properly account for bottom-line identity match performance in the multi-camera setting; (ii) our data set poses realistic challenges to current trackers; and (iii) the performance of our system is comparable to the state of the art.

[1]  Tim J. Ellis,et al.  Bridging the gaps between cameras , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[2]  Mubarak Shah,et al.  A noniterative greedy algorithm for multiframe point correspondence , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Andrew Gilbert,et al.  Tracking Objects Across Cameras by Incrementally Learning Inter-camera Colour Calibration and Patterns of Activity , 2006, ECCV.

[4]  Ramakant Nevatia,et al.  Detection and Tracking of Multiple, Partially Occluded Humans by Bayesian Combination of Edgelet based Part Detectors , 2007, International Journal of Computer Vision.

[5]  Ramakant Nevatia,et al.  Tracking of Multiple, Partially Occluded Humans based on Static Body Part Detection , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[6]  Jean-Marc Odobez,et al.  Multi-Layer Background Subtraction Based on Color and Texture , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Pascal Fua,et al.  Multicamera People Tracking with a Probabilistic Occupancy Map , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Rainer Stiefelhagen,et al.  Evaluating Multiple Object Tracking Performance: The CLEAR MOT Metrics , 2008, EURASIP J. Image Video Process..

[9]  Jinsong Tan,et al.  A note on the inapproximability of correlation clustering , 2007, Inf. Process. Lett..

[10]  Christophe De Vleeschouwer,et al.  Distributed video acquisition and annotation for sport-event summarization , 2008 .

[11]  Ramakant Nevatia,et al.  Global data association for multi-object tracking using network flows , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Mubarak Shah,et al.  Modeling inter-camera space-time and appearance relationships for tracking across non-overlapping views , 2008, Comput. Vis. Image Underst..

[13]  Simone Calderara,et al.  Bayesian-Competitive Consistent Labeling for People Surveillance , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  J. Ferryman,et al.  An overview of the PETS 2009 challenge , 2009 .

[15]  Tiziana D'Orazio,et al.  A Semi-automatic System for Ground Truth Generation of Soccer Video Sequences , 2009, 2009 Sixth IEEE International Conference on Advanced Video and Signal Based Surveillance.

[16]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Irfan A. Essa,et al.  Player localization using multiple static cameras for sports visualization , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Anthony Wirth,et al.  Correlation Clustering , 2010, Encyclopedia of Machine Learning and Data Mining.

[19]  Ramakant Nevatia,et al.  Inter-camera Association of Multi-target Tracks by On-Line Learned Appearance Affinity Models , 2010, ECCV.

[20]  Shishir K. Shah,et al.  Multiple person re-identification using part based spatio-temporal color appearance model , 2011, 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops).

[21]  Vittorio Murino,et al.  Custom Pictorial Structures for Re-identification , 2011, BMVC.

[22]  Yi-Ping Hung,et al.  Adaptive Learning for Target Tracking and True Linking Discovering Across Multiple Non-Overlapping Cameras , 2011, IEEE Transactions on Multimedia.

[23]  Binlong Li,et al.  Dynamic subspace-based coordinated multicamera tracking , 2011, 2011 International Conference on Computer Vision.

[24]  Mohamed R. Amer,et al.  Multiobject tracking as maximum weight independent set , 2011, CVPR 2011.

[25]  Charless C. Fowlkes,et al.  Globally-optimal greedy algorithms for tracking a variable number of objects , 2011, CVPR 2011.

[26]  Pascal Fua,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Multiple Object Tracking Using K-shortest Paths Optimization , 2022 .

[27]  Tieniu Tan,et al.  Direction-based stochastic matching for pedestrian recognition in non-overlapping cameras , 2011, 2011 18th IEEE International Conference on Image Processing.

[28]  Afshin Dehghan,et al.  Part-based multiple-person tracking with partial occlusion handling , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Nathan S. Netanyahu,et al.  A Framework for Inter-camera Association of Multi-target Trajectories by Invariant Target Models , 2012, ACCV Workshops.

[30]  Takeo Kanade,et al.  Computer Vision – ECCV 2012. Workshops and Demonstrations , 2012, Lecture Notes in Computer Science.

[31]  Joachim Denzler,et al.  Data association for multi-object Tracking-by-Detection in multi-camera networks , 2012, 2012 Sixth International Conference on Distributed Smart Cameras (ICDSC).

[32]  Chunxiao Liu,et al.  Person Re-identification: What Features Are Important? , 2012, ECCV Workshops.

[33]  Mubarak Shah,et al.  (MP)2T: Multiple People Multiple Parts Tracker , 2012, ECCV.

[34]  Matej Kristan,et al.  Dana36: A Multi-camera Image Dataset for Object Identification in Surveillance Scenarios , 2012, 2012 IEEE Ninth International Conference on Advanced Video and Signal-Based Surveillance.

[35]  Shishir K. Shah,et al.  Part-based spatio-temporal model for multi-person re-identification , 2012, Pattern Recognit. Lett..

[36]  Afshin Dehghan,et al.  GMCP-Tracker: Global Multi-object Tracking Using Generalized Minimum Clique Graphs , 2012, ECCV.

[37]  Robert T. Collins,et al.  Multiple Target Tracking Using Frame Triplets , 2012, ACCV.

[38]  Robert T. Collins,et al.  Multitarget data association with higher-order motion models , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Liu Li,et al.  Distributed optimization for global data association in non-overlapping camera networks , 2013, 2013 Seventh International Conference on Distributed Smart Cameras (ICDSC).

[40]  Rita Cucchiara,et al.  Learning articulated body models for people re-identification , 2013, MM '13.

[41]  Konrad Schindler,et al.  Challenges of Ground Truth Evaluation of Multi-target Tracking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[42]  Xiaogang Wang,et al.  Unsupervised Salience Learning for Person Re-identification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[43]  Amit K. Roy-Chowdhury,et al.  Information Consensus for Distributed Multi-target Tracking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[44]  Carlo Tomasi,et al.  Tracking Multiple People Online and in Real Time , 2014, ACCV.

[45]  Junjie Yan,et al.  Multiple Target Tracking Based on Undirected Hierarchical Relation Hypergraph , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Kaiqi Huang,et al.  A novel solution for multi-camera object tracking , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[47]  Marco Cristani,et al.  Person Re-identification by Articulated Appearance Matching , 2014, Person Re-Identification.

[48]  Abir Das,et al.  Consistent Re-identification in a Camera Network , 2014, ECCV.

[49]  Bernt Schiele,et al.  Ten Years of Pedestrian Detection, What Have We Learned? , 2014, ECCV Workshops.

[50]  Gian Luca Foresti,et al.  Saliency Weighted Features for Person Re-identification , 2014, ECCV Workshops.

[51]  Gérard G. Medioni,et al.  Exploring context information for inter-camera multiple target tracking , 2014, IEEE Winter Conference on Applications of Computer Vision.

[52]  Guillaume Charpiat,et al.  Multiple Object Tracking by Efficient Graph Partitioning , 2014, ACCV.

[53]  Rita Cucchiara,et al.  Mapping Appearance Descriptors on 3D Body Models for People Re-identification , 2015, International Journal of Computer Vision.

[54]  Xiaojing Chen,et al.  Multitarget Tracking in Nonoverlapping Cameras Using a Reference Set , 2015, IEEE Sensors Journal.

[55]  Afshin Dehghan,et al.  GMMCP tracker: Globally optimal Generalized Maximum Multi Clique problem for multiple object tracking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Stefan Roth,et al.  MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking , 2015, ArXiv.

[57]  Kaiqi Huang,et al.  An equalised global graphical model-based approach for multi-camera object tracking , 2015, ArXiv.

[58]  Bernt Schiele,et al.  Subgraph decomposition for multi-target tracking , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[59]  Amit K. Roy-Chowdhury,et al.  Tracking multiple interacting targets in a camera network , 2015, Comput. Vis. Image Underst..

[60]  Ivan Laptev,et al.  On pairwise costs for network flow multi-object tracking , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[61]  Amit K. Roy-Chowdhury,et al.  A Camera Network Tracking (CamNeT) Dataset and Performance Baseline , 2015, 2015 IEEE Winter Conference on Applications of Computer Vision.

[62]  Deyu Meng,et al.  The Solution Path Algorithm for Identity-Aware Multi-object Tracking , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).