Large-Scale Spatio-Temporal Person Re-Identification: Algorithms and Benchmark

Person re-identification (re-ID) in the scenario with large spatial and temporal spans has not been fully explored. This fact partially occurs because existing benchmark datasets were mainly collected with limited spatial and temporal ranges, e.g., using videos recorded in a few days by cameras in a specific region of the campus. Such limited spatial and temporal ranges make it hard to simulate the difficulties of person re-ID in real scenarios. In this work, we contribute a novel Large-scale Spatio-Temporal (LaST) person re-ID dataset, including 10,862 identities with more than 228k images. Compared with existing datasets, LaST presents more challenging and high-diversity re-ID settings and significantly larger spatial and temporal ranges. For instance, each person can appear in different cities or countries, and in various time slots from day to evening, and in different seasons from spring to winter. To our best knowledge, LaST is a novel person re-ID dataset with the largest spatio-temporal ranges. Based on LaST, we verified its challenge by conducting a comprehensive performance evaluation of 14 re-ID algorithms. We further propose an easy-to-implement baseline that works well in such challenging re-ID settings. We also verified that models pre-trained on LaST can generalize well on existing datasets with short-term and cloth-changing scenarios. We expect LaST to inspire future works toward more realistic and challenging re-ID tasks. More information about the dataset is available at https://github.com/shuxjweb/last.git.

[1]  Yan Lu,et al.  Local Descriptors Optimized for Average Precision , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[2]  Jon Almazán,et al.  Learning With Average Precision: Training Image Retrieval With a Listwise Loss , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[3]  Weihong Deng,et al.  Mixed High-Order Attention Network for Person Re-Identification , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[4]  Linjun Yang,et al.  Embedding-based Retrieval in Facebook Search , 2020, KDD.

[5]  Dahua Lin,et al.  MovieNet: A Holistic Dataset for Movie Understanding , 2020, ECCV.

[6]  Dinesh Kumar Vishwakarma,et al.  A Deep Structure of Person Re-Identification Using Multi-Level Gaussian Models , 2018, IEEE Transactions on Multi-Scale Computing Systems.

[7]  Yang Yang,et al.  ABD-Net: Attentive but Diverse Person Re-Identification , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[8]  Shaogang Gong,et al.  Harmonious Attention Network for Person Re-identification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[9]  Wei Jiang,et al.  Bag of Tricks and a Strong Baseline for Deep Person Re-Identification , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[10]  Feiyue Huang,et al.  Viewpoint-Aware Loss with Angular Regularization for Person Re-Identification , 2019, AAAI.

[11]  Yi Yang,et al.  Hierarchical Temporal Modeling With Mutual Distance Matching for Video Based Person Re-Identification , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Zuozhuo Dai,et al.  Batch DropBlock Network for Person Re-Identification and Beyond , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[14]  Jingsong Xu,et al.  Celebrities-ReID: A Benchmark for Clothes Variation in Long-Term Person Re-Identification , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[15]  Dahua Lin,et al.  Unifying Identification and Context Learning for Person Recognition , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[16]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Zhaoxiang Zhang,et al.  Spectral Feature Transformation for Person Re-Identification , 2018, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[18]  Wei-Shi Zheng,et al.  Person Re-Identification by Contour Sketch Under Moderate Clothing Change , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Chunxiao Liu,et al.  Person re-identification by manifold ranking , 2013, 2013 IEEE International Conference on Image Processing.

[20]  Xiaogang Wang,et al.  Human Reidentification with Transferred Metric Learning , 2012, ACCV.

[21]  Shihua Li,et al.  COCAS: A Large-Scale Clothes Changing Person Dataset for Re-Identification , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Tao Xiang,et al.  Deep Learning for Person Re-Identification: A Survey and Outlook , 2020, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Tao Mei,et al.  Deep Transfer Hashing for Image Retrieval , 2021, IEEE transactions on circuits and systems for video technology (Print).

[24]  Gang Yu,et al.  High-Order Information Matters: Learning Relation and Topology for Occluded Person Re-Identification , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Shaogang Gong,et al.  Faster Person Re-Identification , 2020, ECCV.

[26]  Dinesh Kumar Vishwakarma,et al.  Person Re-Identification using Deep Learning Networks: A Systematic Review , 2020, ArXiv.

[27]  Xiaogang Wang,et al.  DeepReID: Deep Filter Pairing Neural Network for Person Re-identification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Longhui Wei,et al.  Person Transfer GAN to Bridge Domain Gap for Person Re-identification , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[29]  Qi Tian,et al.  Beyond Part Models: Person Retrieval with Refined Part Pooling , 2017, ECCV.

[30]  Vicky S. Kalogeiton,et al.  Smooth-AP: Smoothing the Path Towards Large-Scale Image Retrieval , 2020, ECCV.

[31]  Fei Wang,et al.  Discriminative Feature Learning With Consistent Attention Regularization for Person Re-Identification , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[32]  Andrew Zisserman,et al.  Faces in Places: compound query retrieval , 2016, BMVC.

[33]  Zheng Liu,et al.  Hierarchical Integration of Rich Features for Video-Based Person Re-Identification , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Tao Xiang,et al.  Long-Term Cloth-Changing Person Re-identification , 2020, ACCV.

[35]  Francesco Solera,et al.  Performance Measures and a Data Set for Multi-target, Multi-camera Tracking , 2016, ECCV Workshops.

[36]  Haifeng Hu,et al.  Multiscale Omnibearing Attention Networks for Person Re-Identification , 2021, IEEE Transactions on Circuits and Systems for Video Technology.

[37]  Gian Luca Foresti,et al.  Aggregating Deep Pyramidal Representations for Person Re-Identification , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[38]  Mang Ye,et al.  A Survey of Open-World Person Re-Identification , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[39]  Yanwei Fu,et al.  When Person Re-identification Meets Changing Clothes , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[40]  Xiong Chen,et al.  Learning Discriminative Features with Multiple Granularities for Person Re-Identification , 2018, ACM Multimedia.

[41]  Lei Zhang,et al.  Optimal Projection Guided Transfer Hashing for Image Retrieval , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[42]  Kun He,et al.  Hashing as Tie-Aware Learning to Rank , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[43]  Helio Pedrini,et al.  Top-DB-Net: Top DropBlock for Activation Enhancement in Person Re-Identification , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[44]  Dinesh Kumar Vishwakarma,et al.  State-of-the-Arts Person Re-Identification Using Deep Learning , 2019, 2019 6th International Conference on Signal Processing and Integrated Networks (SPIN).

[45]  Ling Shao,et al.  Interpretable and Generalizable Person Re-identification with Query-Adaptive Convolution and Temporal Lifting , 2019, ECCV.

[46]  Qixiang Ye,et al.  FreeAnchor: Learning to Match Anchors for Visual Object Detection , 2019, NeurIPS.

[47]  Yunchao Wei,et al.  Horizontal Pyramid Matching for Person Re-identification , 2018, AAAI.

[48]  Jianhuang Lai,et al.  Weakly Supervised Person Re-ID: Differentiable Graphical Learning and a New Benchmark , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[49]  Ning Zhang,et al.  Beyond frontal faces: Improving Person Recognition using multiple cues , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Hai Tao,et al.  Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features , 2008, ECCV.

[51]  Qiang Wu,et al.  Beyond Scalar Neuron: Adopting Vector-Neuron Capsules for Long-Term Person Re-Identification , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[52]  Qi Tian,et al.  Scalable Person Re-identification: A Benchmark , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[53]  Yang Song,et al.  Learning Fine-Grained Image Similarity with Deep Ranking , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[54]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[55]  Victor S. Lempitsky,et al.  Learning Deep Embeddings with Histogram Loss , 2016, NIPS.

[56]  Vipin Kumar,et al.  Personalized Image Retrieval with Sparse Graph Representation Learning , 2020, KDD.

[57]  Yi Yang,et al.  Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[58]  Xiaogang Wang,et al.  Joint Detection and Identification Feature Learning for Person Search , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).