Visible-infrared cross-modality person re-identification based on whole-individual training

Abstract Visible-infrared cross-modality person re-identification (VI-ReID) aims to search person images across cameras of different modalities, which can make up for the problem that ReID cannot be performed through visible images in a dark environment. The difficulty of VI-ReID task is the huge discrepancy between the visible modality and the infrared modality. In this paper, a novel whole-individual training (WIT) model is proposed for VI-ReID, which is based on the idea of pulling in the whole and distinguishing the individuals. Specifically, the model is divided into a whole part and an individual part. Two loss functions are developed in the whole part, namely center maximum mean discrepancy (CMMD) loss and intra-class heterogeneous center (ICHC) loss. Ignoring identity difference and treating each modality as a whole, the CMMD loss pulls in the centers of the two modalities. Ignoring modality difference and treating each identify as a whole, the ICHC loss pulls images with the same identity to its cross-modality center. In the individual part, a cross-modality triplet (CMT) loss is employed, which can distinguish the pedestrian images with different identities. The WIT model can help the network identify pedestrian images in an all-round way. Experiments show that the VI-ReID performance of the proposed method is better than existing technologies on two most popular benchmark datasets SYSU-MM01 and RegDB.

[1]  Shijian Lu,et al.  A Similarity Inference Metric for RGB-Infrared Cross-Modality Person Re-identification , 2020, IJCAI.

[2]  Jie Li,et al.  Dual-alignment Feature Embedding for Cross-modality Person Re-identification , 2019, ACM Multimedia.

[3]  Qi Tian,et al.  Beyond Part Models: Person Retrieval with Refined Part Pooling , 2017, ECCV.

[4]  Xiaogang Wang,et al.  CAMP: Cross-Modal Adaptive Message Passing for Text-Image Retrieval , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[5]  Jian Cheng,et al.  Enhancing the Discriminative Feature Learning for Visible-Thermal Cross-Modality Person Re-Identification , 2019, Neurocomputing.

[6]  Xinbo Gao,et al.  Triplet-Based Deep Hashing Network for Cross-Modal Retrieval , 2018, IEEE Transactions on Image Processing.

[7]  Shengcai Liao,et al.  Person re-identification by Local Maximal Occurrence representation and metric learning , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Xiao-Ping Zhang,et al.  Deep learning-based methods for person re-identification: A comprehensive review , 2019, Neurocomputing.

[10]  Bin Liu,et al.  Cross-Modality Person Re-Identification With Shared-Specific Feature Transfer , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Yang Li,et al.  Deep hard modality alignment for visible thermal person re-identification , 2020, Pattern Recognit. Lett..

[12]  Xiaopeng Hong,et al.  Infrared-Visible Cross-Modal Person Re-Identification with an X Modality , 2020, AAAI.

[13]  Changick Kim,et al.  Hi-CMD: Hierarchical Cross-Modality Disentanglement for Visible-Infrared Person Re-Identification , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Xian-Sheng Hua,et al.  Decouple co-adaptation: Classifier randomization for person re-identification , 2020, Neurocomputing.

[15]  Jie Li,et al.  Modality adversarial neural network for visible-thermal person re-identification , 2020, Pattern Recognit..

[16]  Yang Yang,et al.  RGB-Infrared Cross-Modality Person Re-Identification via Joint Pixel and Feature Alignment , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[17]  Dapeng Tao,et al.  Hetero-Center Loss for Cross-Modality Person Re-Identification , 2019, Neurocomputing.

[18]  Wei Liu,et al.  Self-Supervised Adversarial Hashing Networks for Cross-Modal Retrieval , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[19]  Jian Cheng,et al.  Cross-Modality Paired-Images Generation for RGB-Infrared Person Re-Identification , 2020, AAAI.

[20]  Jianhuang Lai,et al.  Learning Modality-Specific Representations for Visible-Infrared Person Re-Identification , 2020, IEEE Transactions on Image Processing.

[21]  Pong C. Yuen,et al.  Hierarchical Discriminative Learning for Visible Thermal Person Re-Identification , 2018, AAAI.

[22]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[23]  Zheng Wang,et al.  Visible Thermal Person Re-Identification via Dual-Constrained Top-Ranking , 2018, IJCAI.

[24]  Zhou Zhao,et al.  Cross-Modal Interaction Networks for Query-Based Moment Retrieval in Videos , 2019, SIGIR.

[25]  Yung-Yu Chuang,et al.  Learning to Reduce Dual-Level Discrepancy for Infrared-Visible Person Re-Identification , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[26]  Yu Qiao,et al.  A Discriminative Feature Learning Approach for Deep Face Recognition , 2016, ECCV.

[27]  Jian Cheng,et al.  Cross-modality paired-images generation and augmentation for RGB-infrared person re-identification , 2020, Neural Networks.

[28]  Jie Li,et al.  HSME: Hypersphere Manifold Embedding for Visible Thermal Person Re-Identification , 2019, AAAI.

[29]  Tien Dat Nguyen,et al.  Person Recognition System Based on a Combination of Body Images from Visible Light and Thermal Cameras , 2017, Sensors.

[30]  Qi Tian,et al.  Scalable Person Re-identification: A Benchmark , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[31]  Jian-Huang Lai,et al.  RGB-Infrared Cross-Modality Person Re-identification , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).