Open-World Person Re-Identification With RGBD Camera in Top-View Configuration for Retail Applications

Person re-identification (re-ID) is currently a notably topic in the computer vision and pattern recognition communities. However, most of the existing works on re-ID have been designed for closed world scenarios, rather than more realistic open world scenarios, limiting the practical application of these re-ID techniques. In a common real-world application, a watch-list of known people is given as the gallery/target set for searching through a large volume of videos where the people on the watch-list are likely to return. This aspect is fundamental in retail for understanding how customers schedule their shopping. The identification of regular and occasional customers allows to define temporal purchasing profiles, which can put in correlation the customers’ temporal habits with other information such as the amount of expenditure and number of purchased items. This paper presents the first attempt to solve a more realistic re-ID setting, designed to face these important issues called Top-View Open-World (TVOW) person re-id. The approach is based on a pretrained Deep Convolutional neural Network (DCNN), finetuned on a dataset acquired by using a top-view configuration. A special loss function called triplet loss was used to train the network. The triplet loss optimizes the embedding space such that data points with the same identity are closer to each other than those with different identities. The TVOW is evaluated on the TVPR2 dataset for people re-ID that is publicly available. The experimental results show that the proposed methods significantly outperform all competitive state-of-the-art methods, bringing to different and significative insights for implicit and extensive shopper behaviour analysis for marketing applications.

[1]  Xiang Li,et al.  Adversarial Open-World Person Re-Identification , 2018, ECCV.

[2]  Li Fei-Fei,et al.  Recurrent Attention Models for Depth-Based Person Identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Emanuele Frontoni,et al.  Person Re-identification Dataset with RGB-D Camera in a Top-View Configuration , 2016, VAAM/FFER@ICPR.

[4]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Emanuele Frontoni,et al.  Robotic retail surveying by deep learning visual and textual data , 2019, Robotics Auton. Syst..

[6]  XiangTao,et al.  Towards Open-World Person Re-Identification by One-Shot Group-Based Verification , 2016 .

[7]  Shaogang Gong,et al.  Reidentification by Relative Distance Comparison , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Horst Bischof,et al.  Relaxed Pairwise Learned Metric for Person Re-identification , 2012, ECCV.

[9]  Shishir K. Shah,et al.  A survey of approaches and trends in person re-identification , 2014, Image Vis. Comput..

[10]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[11]  Thomas B. Moeslund,et al.  Person Re-Identification Using Spatial and Layer-Wise Attention , 2020, IEEE Transactions on Information Forensics and Security.

[12]  Emanuele Frontoni,et al.  Shopper Analytics: A Customer Activity Recognition System Using a Distributed RGB-D Camera Network , 2014, VAAM@ICPR.

[13]  Rongrong Ji,et al.  Pyramidal Person Re-IDentification via Multi-Loss Dynamic Training , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Shaogang Gong,et al.  Towards unsupervised open-set person re-identification , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[15]  Wei-Shi Zheng,et al.  Fast Open-World Person Re-Identification , 2018, IEEE Transactions on Image Processing.

[16]  Bingpeng Ma,et al.  Local Descriptors Encoded by Fisher Vectors for Person Re-identification , 2012, ECCV Workshops.

[17]  Zhuowen Tu,et al.  Feature Mining for Image Classification , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[18]  Shaogang Gong,et al.  Open-world Person Re-Identification by Multi-Label Assignment Inference , 2014, BMVC.

[19]  Emanuele Frontoni,et al.  Convolutional Networks for Semantic Heads Segmentation using Top-View Depth Data in Crowded Environment , 2018, 2018 24th International Conference on Pattern Recognition (ICPR).

[20]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Hai Tao,et al.  Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features , 2008, ECCV.

[22]  Rita Cucchiara,et al.  People reidentification in surveillance and forensics , 2013, ACM Comput. Surv..

[23]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[24]  Shaogang Gong,et al.  Transfer re-identification: From person to set-based verification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[25]  Zhangyang Wang,et al.  In Defense of the Triplet Loss Again: Learning Robust Person Re-Identification with Fast Approximated Triplet Loss and Label Distillation , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[26]  Emanuele Frontoni,et al.  People Detection and Tracking from an RGB-D Camera in Top-View Configuration: Review of Challenges and Applications , 2017, ICIAP Workshops.

[27]  Emanuele Frontoni,et al.  Person Re-Identification with RGB-D Camera in Top-View Configuration through Multiple Nearest Neighbor Classifiers and Neighborhood Component Features Selection , 2018, Sensors.

[28]  Thomas B. Moeslund,et al.  Multimodal Neural Network for Overhead Person Re-Identification , 2017, 2017 International Conference of the Biometrics Special Interest Group (BIOSIG).

[29]  Thomas B. Moeslund,et al.  Attention in Multimodal Neural Networks for Person Re-identification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[30]  Mang Ye,et al.  A Survey of Open-World Person Re-Identification , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Yi Yang,et al.  Person Re-identification: Past, Present and Future , 2016, ArXiv.

[32]  Xiaogang Wang,et al.  Unsupervised Salience Learning for Person Re-identification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Lucas Beyer,et al.  In Defense of the Triplet Loss for Person Re-Identification , 2017, ArXiv.

[34]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Shengcai Liao,et al.  Open-set Person Re-identification , 2014, ArXiv.

[36]  Emanuele Frontoni,et al.  A business application of RTLS technology in Intelligent Retail Environment: Defining the shopper's preferred path and its segmentation , 2019, Journal of Retailing and Consumer Services.

[37]  Alessandro Perina,et al.  Person re-identification by symmetry-driven accumulation of local features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.