Multi-level feature fusion model-based real-time person re-identification for forensics

Person forensics aims to retrieve the specified person across non-overlapping cameras. It is difficult owing to the appearance variations caused by occlusion, human pose change, background clutter, illumination variation, etc. In this scenario, current models face great challenges in extracting effective features. Recent deep learning models mainly focus on extracting representative deep features to cope with appearance variations, while handcrafted features are not fully explored. In this paper, a multi-level feature fusion model (MFFM) is designed to combine both deep features and handcrafted features in real time. MFFM is first utilized to describe person appearance. Then, local binary pattern (LBP) and histogram of oriented gradient (HOG) are extracted to cope with geometric change and illumination variance. Using LBP and HOG, 11.89% on the CUHK03, 15.30% on the Market-1501 and 8.25% on the VIPeR top-1 recognition accuracy improvement for the proposed method are achieved with only 9.66%, 4.90%, and 7.59% extra processing time. Experimental results indicate MFFM can achieve the best performance compared to the state-of-the-art models on the Market1501, CUHK03, and VIPeR datasets.

[1]  Pavlo Molchanov,et al.  Multilayer and Multimodal Fusion of Deep Neural Networks for Video Classification , 2016, ACM Multimedia.

[2]  Victor Lempitsky,et al.  Multiregion Bilinear Convolutional Neural Networks for Person Re-Identification , 2015 .

[3]  Qiang He,et al.  Time-aware distributed service recommendation with privacy-preservation , 2019, Inf. Sci..

[4]  Xiao Liu,et al.  Semi-supervised Coupled Dictionary Learning for Person Re-identification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Wei Li,et al.  Transferable Joint Attribute-Identity Deep Learning for Unsupervised Person Re-identification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[6]  Lin Wu,et al.  PersonNet: Person Re-identification with Deep Convolutional Neural Networks , 2016, ArXiv.

[7]  Xiaogang Wang,et al.  Learning Mid-level Filters for Person Re-identification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Shiliang Zhang,et al.  Deep Attributes Driven Multi-Camera Person Re-identification , 2016, ECCV.

[9]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Sergio A. Velastin,et al.  Local Fisher Discriminant Analysis for Pedestrian Re-identification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Yi Yang,et al.  Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[12]  Alessandro Perina,et al.  Person re-identification by symmetry-driven accumulation of local features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[13]  Shengcai Liao,et al.  Salient Color Names for Person Re-identification , 2014, ECCV.

[14]  Jesús Martínez del Rincón,et al.  Person Reidentification Using Deep Convnets With Multitask Learning , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[15]  Haibin Ling,et al.  Multi-Level Contextual RNNs With Attention Model for Scene Labeling , 2016, IEEE Transactions on Intelligent Transportation Systems.

[16]  Shengcai Liao,et al.  Efficient PSD Constrained Asymmetric Metric Learning for Person Re-Identification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[17]  Lei Zhang,et al.  Higher-Order Integration of Hierarchical Convolutional Activations for Fine-Grained Visual Categorization , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[18]  Jitendra Malik,et al.  Hypercolumns for object segmentation and fine-grained localization , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[19]  Xiaogang Wang,et al.  Unsupervised Salience Learning for Person Re-identification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Zhen Li,et al.  Learning Locally-Adaptive Decision Functions for Person Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  Shuicheng Yan,et al.  End-to-End Comparative Attention Networks for Person Re-Identification , 2016, IEEE Transactions on Image Processing.

[22]  Liang Zheng,et al.  Improving Person Re-identification by Attribute and Identity Learning , 2017, Pattern Recognit..

[23]  Qi Cui.  Identifying materials of photographic images and photorealistic computer generated graphics based on deep CNNs , 2018 .

[24]  Nanning Zheng,et al.  Person Re-identification by Multi-Channel Parts-Based CNN with Improved Triplet Loss Function , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Xiaogang Wang,et al.  Locally Aligned Feature Transforms across Views , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Hai Tao,et al.  Evaluating Appearance Models for Recognition, Reacquisition, and Tracking , 2007 .

[27]  Zhili Zhou,et al.  Multiple Distance-Based Coding: Toward Scalable Feature Matching for Large-Scale Web Image Search , 2019, IEEE Transactions on Big Data.

[28]  Victor S. Lempitsky,et al.  Multi-Region bilinear convolutional neural networks for person re-identification , 2015, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[29]  Xuyun Zhang,et al.  A QoS-aware virtual machine scheduling method for energy conservation in cloud-based cyber-physical systems , 2019, World Wide Web.

[30]  Shaogang Gong,et al.  Learning a Discriminative Null Space for Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Ruohan Meng.  A fusion steganographic algorithm based on Faster R-CNN , 2018 .

[32]  Jian Dong,et al.  Collaborative Layer-Wise Discriminative Learning in Deep Neural Networks , 2016, ECCV.

[33]  Gang Wang,et al.  Gated Siamese Convolutional Neural Network Architecture for Human Re-identification , 2016, ECCV.

[34]  Xiaogang Wang,et al.  Spindle Net: Person Re-identification with Human Body Region Guided Feature Decomposition and Fusion , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Rainer Stiefelhagen,et al.  Person Re-identification by Deep Learning Attribute-Complementary Information , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[36]  Nanning Zheng,et al.  Similarity learning on an explicit polynomial kernel feature map for person re-identification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Nanning Zheng,et al.  Similarity Learning with Spatial Constraints for Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Christoph Adami.  Artificial intelligence: Robots with instincts , 2015, Nature.

[39]  Xiaogang Wang,et al.  DeepReID: Deep Filter Pairing Neural Network for Person Re-identification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Zhiguo Qu,et al.  A Novel Quantum Stegonagraphy Based on Brown States , 2018 .

[41]  Bingpeng Ma,et al.  BiCov: a novel image representation for person re-identification and face verification , 2012, BMVC.

[42]  Xiaogang Wang,et al.  Person Re-identification by Salience Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[43]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Songfan Yang,et al.  Multi-scale Recognition with DAG-CNNs , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[45]  Tao Xiang,et al.  Transferring a semantic representation for person re-identification and search , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[46]  Xiaogang Wang,et al.  HydraPlus-Net: Attentive Deep Features for Pedestrian Analysis , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[47]  Tetsu Matsukawa,et al.  Person re-identification using CNN features learned from combination of attributes , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[48]  Shaogang Gong,et al.  Harmonious Attention Network for Person Re-identification , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[49]  Shengcai Liao,et al.  Person re-identification by Local Maximal Occurrence representation and metric learning , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[50]  Hongxun Yao,et al.  Exploiting the complementary strengths of multi-layer CNN features for image retrieval , 2017, Neurocomputing.

[51]  Cordelia Schmid,et al.  Learning Color Names for Real-World Applications , 2009, IEEE Transactions on Image Processing.

[52]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[53]  Qi Tian,et al.  Scalable Person Re-identification: A Benchmark , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[54]  Takahiro Okabe,et al.  Hierarchical Gaussian Descriptor for Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[55]  Abir Das,et al.  Consistent Re-identification in a Camera Network , 2014, ECCV.

[56]  Larry S. Davis,et al.  Joint Learning for Attribute-Consistent Person Re-Identification , 2014, ECCV Workshops.

[57]  Jinjun Chen,et al.  A two-stage locality-sensitive hashing based approach for privacy-preserving mobile service recommendation in cross-platform edge environment , 2018, Future Gener. Comput. Syst..