City-Scale Multi-Camera Vehicle Tracking by Semantic Attribute Parsing and Cross-Camera Tracklet Matching

This paper focuses on the Multi-Target Multi-Camera Tracking (MTMCT) task in a city-scale multi-camera network. As the trajectory of each target is naturally split into multiple sub-trajectories (namely local tracklets) in different cameras, the key issue of MTMCT is how to match local tracklets belonging to the same target across different cameras. To this end, we propose an efficient two-step MTMCT approach to robustly track vehicles in a camera network. It first generates all local tracklets and then matches the ones belonging to the same target across different cameras. More specifically, in the local tracklet generation phase, we follow the tracking-by-detection paradigm and link the detections to local tracklets by graph clustering. In the cross-camera tracklet matching phase, we first develop a spatial-temporal attention mechanism to produce robust tracklet representations. We then prune false matching candidates by traffic topology reasoning and match tracklets across cameras using the recently proposed TRACklet-to-Target Assignment (TRACTA) algorithm. The proposed method is evaluated on the City-Scale Multi-Camera Vehicle Tracking task at the 2020 AI City Challenge and achieves the second-best results.

[1]  K. R. Ramakrishnan,et al.  Distributed Person of Interest Tracking In Camera Networks , 2017, ICDSC.

[2]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[4]  Yihong Gong,et al.  Multi-Target Multi-Camera Tracking by Tracklet-to-Target Assignment , 2020, IEEE Transactions on Image Processing.

[5]  Wei Jiang,et al.  Bag of Tricks and a Strong Baseline for Deep Person Re-Identification , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[6]  Mei Han,et al.  An algorithm for multiple object trajectory tracking , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[7]  Wei Wu,et al.  Online Inter-Camera Trajectory Association Exploiting Person Re-Identification and Camera Topology , 2018, ACM Multimedia.

[8]  Afshin Dehghan,et al.  GMCP-Tracker: Global Multi-object Tracking Using Generalized Minimum Clique Graphs , 2012, ECCV.

[9]  Pascal Fua,et al.  Multicamera People Tracking with a Probabilistic Occupancy Map , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Bernt Schiele,et al.  Multiple People Tracking by Lifted Multicut and Person Re-identification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Francesco Solera,et al.  Performance Measures and a Data Set for Multi-target, Multi-camera Tracking , 2016, ECCV Workshops.

[12]  Joachim Denzler,et al.  Data association for multi-object Tracking-by-Detection in multi-camera networks , 2012, 2012 Sixth International Conference on Distributed Smart Cameras (ICDSC).

[13]  Silvio Savarese,et al.  Recurrent Autoregressive Networks for Online Multi-object Tracking , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[14]  James M. Rehg,et al.  Multiple Hypothesis Tracking Revisited , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[15]  Silvio Savarese,et al.  Learning to Track: Online Multi-object Tracking by Decision Making , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[16]  Yang Liu,et al.  Multi-view People Tracking via Hierarchical Trajectory Composition , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Jenq-Neng Hwang,et al.  Multi-Camera Tracking of Vehicles based on Deep Features Re-ID and Trajectory-Based Camera Link Models , 2019, CVPR Workshops.

[18]  Hua Yang,et al.  Online Multi-Object Tracking with Dual Matching Attention Networks , 2018, ECCV.

[19]  Wei Jiang,et al.  A Strong Baseline and Batch Normalization Neck for Deep Person Re-Identification , 2019, IEEE Transactions on Multimedia.

[20]  Daniel Wolf,et al.  Hypergraphs for Joint Multi-view Reconstruction and Multi-object Tracking , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Mubarak Shah,et al.  Tracking in uncalibrated cameras with overlapping field of view , 2001 .

[22]  Haibin Ling,et al.  Online Multi-Object Tracking With Instance-Aware Tracker and Dynamic Model Refreshment , 2019, 2019 IEEE Winter Conference on Applications of Computer Vision (WACV).

[23]  Jenq-Neng Hwang,et al.  Online-Learning-Based Human Tracking Across Non-Overlapping Cameras , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[24]  Nanning Zheng,et al.  Part-aware trajectories association across non-overlapping uncalibrated cameras , 2017, Neurocomputing.

[25]  Silvio Savarese,et al.  Tracking the Untrackable: Learning to Track Multiple Cues with Long-Term Dependencies , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[26]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Kaiqi Huang,et al.  An Equalized Global Graph Model-Based Approach for Multicamera Object Tracking , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[28]  Rui Yao,et al.  Model-Free Tracker for Multiple Objects Using Joint Appearance and Motion Inference , 2020, IEEE Transactions on Image Processing.

[29]  Kaiqi Huang,et al.  A novel solution for multi-camera object tracking , 2014, 2014 IEEE International Conference on Image Processing (ICIP).

[30]  Jenq-Neng Hwang,et al.  On-Road Pedestrian Tracking Across Multiple Driving Recorders , 2015, IEEE Transactions on Multimedia.

[31]  Pascal Fua,et al.  Multi-Commodity Network Flow for Tracking Multiple People , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Luc Van Gool,et al.  Customized Multi-person Tracker , 2018, ACCV.

[33]  J. Munkres.  ALGORITHMS FOR THE ASSIGNMENT AND TRANSIORTATION tROBLEMS* , 1957 .

[34]  Yael Moses,et al.  Homography based multiple camera detection and tracking of people in a dense crowd , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[35]  Aubrey B. Poore,et al.  Multidimensional assignment formulation of data association problems arising from multitarget and multisensor tracking , 1994, Comput. Optim. Appl..

[36]  James M. Rehg,et al.  Multi-object Tracking with Neural Gating Using Bilinear LSTM , 2018, ECCV.

[37]  Wei Xu,et al.  Tracklet Association Tracker: An End-to-End Learning-based Association Approach for Multi-Object Tracking , 2018, ArXiv.

[38]  Song-Chun Zhu,et al.  Cross-View People Tracking by Scene-Centered Spatio-Temporal Parsing , 2017, AAAI.

[39]  Sergey Ioffe,et al.  Rethinking the Inception Architecture for Computer Vision , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[41]  Yifei Zhang,et al.  Spatio-temporal Consistency and Hierarchical Matching for Multi-Target Multi-Camera Vehicle Tracking , 2019, CVPR Workshops.

[42]  Marcello Pelillo,et al.  Multi-target Tracking in Multiple Non-overlapping Cameras Using Fast-Constrained Dominant Sets , 2019, International Journal of Computer Vision.

[43]  Emanuele Frontoni,et al.  Shopper Analytics: A Customer Activity Recognition System Using a Distributed RGB-D Camera Network , 2014, VAAM@ICPR.

[44]  Tieniu Tan,et al.  Principal axis-based correspondence between multiple cameras for people tracking , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[45]  Pascal Fua,et al.  Ieee Transactions on Pattern Analysis and Machine Intelligence 1 Multiple Object Tracking Using K-shortest Paths Optimization , 2022 .

[46]  Carlo Tomasi,et al.  Features for Multi-target Multi-camera Tracking and Re-identification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[47]  Jenq-Neng Hwang,et al.  CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[48]  Dietrich Paulus,et al.  Simple online and realtime tracking with a deep association metric , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[49]  Yu Liu,et al.  POI: Multiple Object Tracking with High Performance Detection and Appearance Feature , 2016, ECCV Workshops.

[50]  Ingemar J. Cox,et al.  An Efficient Implementation of Reid's Multiple Hypothesis Tracking Algorithm and Its Evaluation for the Purpose of Visual Tracking , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Jun Li,et al.  Hierarchical Tracking by Reinforcement Learning-Based Searching and Coarse-to-Fine Verifying , 2019, IEEE Transactions on Image Processing.

[52]  Gérard G. Medioni,et al.  Exploring context information for inter-camera multiple target tracking , 2014, IEEE Winter Conference on Applications of Computer Vision.

[53]  Fabio Tozeto Ramos,et al.  Simple online and realtime tracking , 2016, 2016 IEEE International Conference on Image Processing (ICIP).