Large Scale Vehicle Re-Identification by Knowledge Transfer from Simulated Data and Temporal Attention

Automated re-identification (re-id) of vehicles is the foundation of many traffic analysis applications across camera networks, e.g. vehicle tracking, counting, or traffic density and flow estimation. The re-id task is made difficult by variations in lighting, viewpoint, image quality and similar vehicle models and colors that can occur across the network. These influences can cause a high visual appearance variation for the same vehicle while different vehicles may appear near identical under similar conditions. However, with a growing number of available datasets and well crafted deep learning models, much progress has been made. In this work we summarize the results of our participation in the NVIDIA AI City Challenge 2020 for vehicle reid. We address the re-id task by relying on well-proven design choices from the closely related person re-id literature. In addition to this, we explicitly address viewpoint and occlusions variation. The former is addressed by incorporating vehicle viewpoint classification results into our matching distance. The required viewpoint classifier is trained predominantly on simulated data and we show that it can be applied to real-world imagery with minimal domain adaptation. We address occlusion by relying on temporal attention scores which emphasize video frames that contain minimal occlusion. Finally, we further boost re-id accuracy by applying video-based re-ranking and an ensemble of complementary models. Our models, code, and simulated data is available at https://github.com/corner100/2020-aicitychallenge-IOSB-VeRi.

[1]  Rama Chellappa,et al.  Attention Driven Vehicle Re-identification and Unsupervised Anomaly Detection for Traffic Understanding , 2019, CVPR Workshops.

[2]  Ling Shao,et al.  Viewpoint-Aware Attentive Multi-view Inference for Vehicle Re-identification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[3]  Liang Zheng,et al.  Re-ranking Person Re-identification with k-Reciprocal Encoding , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Yi Yang,et al.  VehicleNet: Learning Robust Feature Representation for Vehicle Re-identification , 2019, CVPR Workshops.

[5]  Francois Bremond,et al.  Partition and Reunion: A Two-Branch Neural Network for Vehicle Re-identification , 2019, CVPR Workshops.

[6]  M. Saquib Sarfraz,et al.  A Pose-Sensitive Embedding for Person Re-identification with Expanded Cross Neighborhood Re-ranking , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[7]  Xiaogang Wang,et al.  Learning Deep Neural Networks for Vehicle Re-ID with Visual-spatio-Temporal Path Proposals , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[8]  Liang Zheng,et al.  Simulating Content Consistent Vehicle Datasets with Attribute Descent , 2019, ECCV.

[9]  Wei Wu,et al.  Multi-Camera Vehicle Tracking with Powerful Visual Features and Spatial-Temporal Cue , 2019, CVPR Workshops.

[10]  Wei Jiang,et al.  Bag of Tricks and a Strong Baseline for Deep Person Re-Identification , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[11]  Tao Xiang,et al.  Deep Learning for Person Re-Identification: A Survey and Outlook , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Shengcai Liao,et al.  Vehicle Re-Identification Using Quadruple Directional Deep Learning Features , 2018, IEEE Transactions on Intelligent Transportation Systems.

[13]  Zheng Wang,et al.  Person Reidentification via Ranking Aggregation of Similarity Pulling and Dissimilarity Pushing , 2016, IEEE Transactions on Multimedia.

[14]  Jenq-Neng Hwang,et al.  CityFlow: A City-Scale Benchmark for Multi-Target Multi-Camera Vehicle Tracking and Re-Identification , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Shuo Wang,et al.  PAMTRI: Pose-Aware Multi-Task Learning for Vehicle Re-Identification Using Highly Randomized Synthetic Data , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[16]  Kai Lv,et al.  Vehicle Re-Identification with Location and Time Stamps , 2019, CVPR Workshops.

[17]  Yi Yang,et al.  Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[18]  Jenq-Neng Hwang,et al.  Multi-View Vehicle Re-Identification using Temporal Attention Model and Metadata Re-ranking , 2019, CVPR Workshops.

[19]  Xiaogang Wang,et al.  Orientation Invariant Feature Embedding and Spatial Temporal Regularization for Vehicle Re-identification , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[20]  Xiao Tan,et al.  Multi-camera vehicle tracking and re-identification based on visual and spatial-temporal features , 2019, CVPR Workshops.