A Multi-Camera Vehicle Tracking System based on City-Scale Vehicle Re-ID and Spatial-Temporal Information

With the demands of the intelligent city and city-scale traffic management, city-scale multi-camera vehicle tracking (MCVT) has become a vital problem. The MCVT is challenging due to frequent occlusion, similar vehicle models, significant feature variation by different lighting conditions, and viewing perspective in different cameras. This paper proposes an MCVT system composed of single-camera tracking (SCT), vehicle re-identification (Re-ID), and multi-camera tracks matching (MCTM). In the SCT phase, we designed a tracker update strategy and used the Re-ID model in advance. We also adopted a template matching method to re-associate the discontinuous tracklets. As for vehicle Re-ID, we implemented a spatial attention mechanism based on the background model. Then we fully leveraged the labels of synthetic data to train attributes Re-ID models as the attributes features extractor. Finally, we proposed an MCTM method to leverage tracklets representation and spatial-temporal information efficiently. Our system is evaluated both on the City-Scale Multi-Camera Vehicle Re-Identification task (Track 2) and City-Scale Multi-Camera Vehicle Tracking task (Track 3) at the AI City Challenge. Our vehicle Re-ID method has achieved 3rd place of Track 2, with an mAP score of 66.50%, and achieved state-of-the-art results on the VeRi776 dataset. Our MCVT system has achieved 3rd place, yielding 76.51% IDF1 of Track 3. Experimental results demonstrate that our system has achieved competitive performance for city-scale traffic management.

[1]  Yichen Wei,et al.  Circle Loss: A Unified Perspective of Pair Similarity Optimization , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Qingming Huang,et al.  Parsing-Based View-Aware Embedding Network for Vehicle Re-Identification , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Jenq-Neng Hwang,et al.  Multi-Camera Tracking of Vehicles based on Deep Features Re-ID and Trajectory-Based Camera Link Models , 2019, CVPR Workshops.

[4]  Tao Mei,et al.  PROVID: Progressive and Multimodal Vehicle Reidentification for Large-Scale Urban Surveillance , 2018, IEEE Transactions on Multimedia.

[5]  Hong-Yuan Mark Liao,et al.  YOLOv4: Optimal Speed and Accuracy of Object Detection , 2020, ArXiv.

[6]  Bing He,et al.  Part-Regularized Near-Duplicate Vehicle Re-Identification , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Wei Jiang,et al.  Bag of Tricks and a Strong Baseline for Deep Person Re-Identification , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[8]  Wenjun Zeng,et al.  Uncertainty-Aware Multi-Shot Knowledge Distillation for Image-Based Object Re-Identification , 2020, AAAI.

[9]  Yihong Gong,et al.  City-Scale Multi-Camera Vehicle Tracking by Semantic Attribute Parsing and Cross-Camera Tracklet Matching , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[10]  Francesco Solera,et al.  Performance Measures and a Data Set for Multi-target, Multi-camera Tracking , 2016, ECCV Workshops.

[11]  T. Başar,et al.  A New Approach to Linear Filtering and Prediction Problems , 2001 .

[12]  Hei Law,et al.  CornerNet: Detecting Objects as Paired Keypoints , 2018, ECCV.

[13]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[14]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Wu Liu,et al.  Large-scale vehicle re-identification in urban surveillance videos , 2016, 2016 IEEE International Conference on Multimedia and Expo (ICME).

[16]  Jenq-Neng Hwang,et al.  CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Yizhou Wang,et al.  Traffic-Aware Multi-Camera Tracking of Vehicles Based on ReID and Camera Link Model , 2020, ACM Multimedia.

[18]  Rama Chellappa,et al.  A Dual-Path Model With Adaptive Attention for Vehicle Re-Identification , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[19]  Ling-Yu Duan,et al.  VERI-Wild: A Large Dataset and a New Method for Vehicle Re-Identification in the Wild , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Harold W. Kuhn,et al.  The Hungarian method for the assignment problem , 1955, 50 Years of Integer Programming.

[21]  J. J. van Vaals,et al.  “Keyhole” method for accelerating imaging of contrast agent uptake , 1993, Journal of magnetic resonance imaging : JMRI.

[22]  Shuo Wang,et al.  PAMTRI: Pose-Aware Multi-Task Learning for Vehicle Re-Identification Using Highly Randomized Synthetic Data , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[23]  Abhinav Gupta,et al.  Non-local Neural Networks , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  Lucas Beyer,et al.  In Defense of the Triplet Loss for Person Re-Identification , 2017, ArXiv.

[25]  R. Chellappa,et al.  The Devil is in the Details: Self-Supervised Attention for Vehicle Re-Identification , 2020, ECCV.

[26]  Fabio Tozeto Ramos,et al.  Simple online and realtime tracking , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[27]  Ross B. Girshick,et al.  Mask R-CNN , 2017, 1703.06870.

[28]  Xiaogang Wang,et al.  Orientation Invariant Feature Embedding and Spatial Temporal Regularization for Vehicle Re-identification , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Nuno Vasconcelos,et al.  Cascade R-CNN: Delving Into High Quality Object Detection , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[30]  Qi Tian,et al.  Scalable Person Re-identification: A Benchmark , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[31]  Xiaogang Wang,et al.  Learning Deep Neural Networks for Vehicle Re-ID with Visual-spatio-Temporal Path Proposals , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[32]  Quoc V. Le,et al.  EfficientDet: Scalable and Efficient Object Detection , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Qiang Wang,et al.  End-to-End Temporal Feature Aggregation for Siamese Trackers , 2020, 2020 IEEE International Conference on Image Processing (ICIP).

[34]  Wenjun Zeng,et al.  FairMOT: On the Fairness of Detection and Re-Identification in Multiple Object Tracking. , 2020 .

[35]  Jenq-Neng Hwang,et al.  Multi-View Vehicle Re-Identification using Temporal Attention Model and Metadata Re-ranking , 2019, CVPR Workshops.

[36]  M. Naphade,et al.  Simulating Content Consistent Vehicle Datasets with Attribute Descent , 2019, ECCV.

[37]  Dietrich Paulus,et al.  Simple online and realtime tracking with a deep association metric , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[38]  Xingyi Zhou,et al.  Objects as Points , 2019, ArXiv.

[39]  Xiaoou Tang,et al.  Two at Once: Enhancing Learning and Generalization Capacities via IBN-Net , 2018, ECCV.

[40]  Yifei Zhang,et al.  Spatio-temporal Consistency and Hierarchical Matching for Multi-Target Multi-Camera Vehicle Tracking , 2019, CVPR Workshops.

[41]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[42]  Francisco Herrera,et al.  Deep Learning in Video Multi-Object Tracking: A Survey , 2019, Neurocomputing.

[43]  Quoc V. Le,et al.  EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks , 2019, ICML.

[44]  Alexander G. Hauptmann,et al.  ELECTRICITY: An Efficient Multi-camera Vehicle Tracking System for Intelligent City , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[45]  Suk-Ju Kang,et al.  Towards Real-time Multi-object Tracking in CPU Environment , 2020 .