Deep-ReID: deep features and autoencoder assisted image patching strategy for person re-identification in smart cities surveillance

Person Re-identification (P-ReID) task searches for true matches of a given query from a large repository of non-overlapping camera’s images/videos. In smart cities surveillance, P-ReID is challenging due to variation in human’s appearance, illumination affects, and difference in viewpoints. The mainstream approaches achieve P-ReID by implementing supervised learning strategies, requiring exhaustive manual annotation, which is probably erroneous due to human involvement. In addition, the employed methods use high-dimensional feature maps to identify a person, which is not a realistic approach in terms of storage resources and computational complexity. To tackle these issues, we incorporate learned features and deep autoencoder under the umbrella of a unified framework for P-ReID. First, we apply a unique image patching strategy by dividing the input image into two parts (upper and lower) and acquire learned features from fully connected layer of a pretrained Convolutional Neural Network (CNN) model for both patches. To achieve efficient and high performance, the proposed framework utilizes a self-tuned autoencoder to acquire low-dimensional representative features. The obtained features are matched with the patterns of database via cosine similarity measurement to re-identify a person’s appearance. The proposed framework provides a trade-off between time complexity and accuracy, where a lightweight model can be incorporated with reduced number of autoencoder layers to obtain fast and comparatively flexible results. The major novelty of the proposed framework includes implementation of a hybrid network mechanism for P-ReID, which shows convincing real-time results and best fits for smart cities surveillance. The proposed framework is tested over several P-ReID datasets to prove its influence over the existing works with reduced computational complexity.

[1]  Miaojing Shi,et al.  A Selective Biogeography-Based Optimizer Considering Resource Allocation for Large-Scale Global Optimization , 2019, Comput. Intell. Neurosci..

[2]  Fawad,et al.  Person Re-Identification by Discriminative Local Features of Overlapping Stripes , 2020, Symmetry.

[3]  Forrest N. Iandola,et al.  SqueezeNet: AlexNet-level accuracy with 50x fewer parameters and <1MB model size , 2016, ArXiv.

[4]  Yaohua Wang,et al.  Hierarchical Clustering With Hard-Batch Triplet Loss for Person Re-Identification , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Nick Barnes,et al.  UC-Net: Uncertainty Inspired RGB-D Saliency Detection via Conditional Variational Autoencoders , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Yi Yang,et al.  Learning to Adapt Invariance in Memory for Person Re-Identification , 2019, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Sung Wook Baik,et al.  Action recognition using optimized deep autoencoder and CNN for surveillance data streams of non-stationary environments , 2019, Future Gener. Comput. Syst..

[8]  Samee Ullah Khan,et al.  MPPIF-Net: Identification of Plasmodium Falciparum Parasite Mitochondrial Proteins Using Deep Features with Multilayer Bi-directional LSTM , 2020, Processes.

[9]  Yi Yang,et al.  A Discriminatively Learned CNN Embedding for Person Reidentification , 2016, ACM Trans. Multim. Comput. Commun. Appl..

[10]  Laurence T. Yang,et al.  An Efficient Deep Learning Model to Predict Cloud Workload for Industry Informatics , 2018, IEEE Transactions on Industrial Informatics.

[11]  Xuelong Li,et al.  Person Re-Identification by Regularized Smoothing KISS Metric Learning , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Min Chen,et al.  Selective transfer cycle GAN for unsupervised person re-identification , 2020, Multimedia Tools and Applications.

[13]  Zheng-Jun Zha,et al.  Real-World Person Re-Identification via Degradation Invariance Learning , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Ping Chen,et al.  Person re-identification based on multi-scale convolutional network , 2019, Multimedia Tools and Applications.

[15]  Lei Wu,et al.  Learning refined attribute-aligned network with attribute selection for person re-identification , 2020, Neurocomputing.

[16]  Bin Song,et al.  CGAN-TM: A Novel Domain-to-Domain Transferring Method for Person Re-Identification , 2020, IEEE Transactions on Image Processing.

[17]  Di Wu,et al.  A deep model with combined losses for person re-identification , 2019, Cognitive Systems Research.

[18]  Sung Wook Baik,et al.  Personalized Movie Summarization Using Deep CNN-Assisted Facial Expression Recognition , 2019, Complex..

[19]  Muhammad Moazam Fraz,et al.  Person Re-Identification Using Hybrid Representation Reinforced by Metric Learning , 2018, IEEE Access.

[20]  Khan Muhammad,et al.  A Novel Deep Transfer Learning Method for Detection of Myocardial Infarction , 2019, ArXiv.

[21]  Luca Bertinetto,et al.  End-to-End Representation Learning for Correlation Filter Based Tracking , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Fengyuan Wang,et al.  Engineering Hand-designed and Deeply-learned features for person Re-identification , 2020, Pattern Recognit. Lett..

[23]  Abhir Bhalerao,et al.  Person reidentification using deep foreground appearance modeling , 2018, J. Electronic Imaging.

[24]  Sung Wook Baik,et al.  Conflux LSTMs Network: A Novel Approach for Multi-View Action Recognition , 2020, Neurocomputing.

[25]  Victor S. Lempitsky,et al.  Multi-Region bilinear convolutional neural networks for person re-identification , 2015, 2017 14th IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS).

[26]  Liang Lin,et al.  Deep feature learning with relative distance comparison for person re-identification , 2015, Pattern Recognit..

[27]  Sung Wook Baik,et al.  Multiview Summarization and Activity Recognition Meet Edge Computing in IoT Environments , 2021, IEEE Internet of Things Journal.

[28]  Jingdong Wang,et al.  Deeply-Learned Part-Aligned Representations for Person Re-identification , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[29]  Guanghui Wang,et al.  Object Detection with Convolutional Neural Networks , 2019, Deep Learning in Computer Vision.

[30]  Xiang Li,et al.  An enhanced deep feature representation for person re-identification , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[31]  R. R. Aliev,et al.  Artificial Neural Networks , 2001 .

[32]  Meng Wang,et al.  Multimodal Deep Autoencoder for Human Pose Recovery , 2015, IEEE Transactions on Image Processing.

[33]  Şaban Öztürk,et al.  Stacked auto-encoder based tagging with deep features for content-based medical image retrieval , 2020, Expert Syst. Appl..

[34]  Jun Yu,et al.  Semantic preserving distance metric learning and applications , 2014, Inf. Sci..

[35]  Xiao-Yuan Jing,et al.  Video-Based Person Re-Identification by Simultaneously Learning Intra-Video and Inter-Video Distance Metrics , 2016, IEEE Transactions on Image Processing.

[36]  Qing Liu,et al.  Multi-Information Flow CNN and Attribute-Aided Reranking for Person Reidentification , 2019, Comput. Intell. Neurosci..

[37]  Xiaogang Wang,et al.  Spindle Net: Person Re-identification with Human Body Region Guided Feature Decomposition and Fusion , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[38]  Nanning Zheng,et al.  Point to Set Similarity Based Deep Feature Learning for Person Re-Identification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Yi Yang,et al.  Unsupervised Person Re-identification via Cross-Camera Similarity Exploration , 2020, IEEE Transactions on Image Processing.

[40]  Nanning Zheng,et al.  Person Re-identification by Multi-Channel Parts-Based CNN with Improved Triplet Loss Function , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Sung Wook Baik,et al.  A comprehensive survey of multi-view video summarization , 2021, Pattern Recognit..

[42]  Zhedong Zheng,et al.  Joint Discriminative and Generative Learning for Person Re-Identification , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Kaiqi Huang,et al.  Beyond Triplet Loss: A Deep Quadruplet Network for Person Re-identification , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[44]  Shaogang Gong,et al.  Learning a Discriminative Null Space for Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Christophe Garcia,et al.  text Detection with Convolutional Neural Networks , 2008, VISAPP.

[46]  Michael Jones,et al.  An improved deep learning architecture for person re-identification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[47]  Jiebo Luo,et al.  Structure alignment of attributes and visual features for cross-dataset person re-identification , 2020, Pattern Recognit..

[48]  Qijun Zhao,et al.  Hierarchical Feature Aggregation from Body Parts for Misalignment Robust Person Re-Identification , 2019 .

[49]  Jingsong Xu,et al.  Deep Learning for Person Reidentification Using Support Vector Machines , 2017, Adv. Multim..

[50]  Qijun Zhao,et al.  Deepside: A general deep framework for salient object detection , 2019, Neurocomputing.

[51]  Shin'ichi Satoh,et al.  Person Reidentification via Discrepancy Matrix and Matrix Metric , 2018, IEEE Transactions on Cybernetics.

[52]  Ling Shao,et al.  See More, Know More: Unsupervised Video Object Segmentation With Co-Attention Siamese Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Lei Liu,et al.  Multi-level feature fusion model-based real-time person re-identification for forensics , 2019, Journal of Real-Time Image Processing.

[54]  Qiuqi Ruan,et al.  View-specific subspace learning and re-ranking for semi-supervised person re-identification , 2020, Pattern Recognit..

[55]  Sung Wook Baik,et al.  Deep LSTM-Based Sequence Learning Approaches for Action and Activity Recognition , 2020 .

[56]  Lin Du,et al.  Joint Attention Mechanism for Person Re-Identification , 2019, IEEE Access.

[57]  Qi Tian,et al.  Scalable Person Re-identification: A Benchmark , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[58]  François Fleuret,et al.  Scalable Metric Learning via Weighted Approximate Rank Component Analysis , 2016, ECCV.

[59]  Lin Wu,et al.  Deep Linear Discriminant Analysis on Fisher Networks: A Hybrid Architecture for Person Re-identification , 2016, Pattern Recognit..

[60]  Hao Liu,et al.  Local region partition for person re-identification , 2019, Multimedia Tools and Applications.

[61]  Wenguan Wang,et al.  Shifting More Attention to Video Salient Object Detection , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[62]  Sung Wook Baik,et al.  Cover the Violence: A Novel Deep-Learning-Based Approach Towards Violence-Detection in Movies , 2019, Applied Sciences.

[63]  Jian-Huang Lai,et al.  Person Re-Identification by Camera Correlation Aware Feature Augmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[64]  Xiaogang Wang,et al.  Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  Abdulrahman A. Alshdadi,et al.  Gaussian process regression (GPR) based non-invasive continuous blood pressure prediction method from cuff oscillometric signals , 2020 .

[66]  Barbara Caputo,et al.  Looking beyond appearances: Synthetic training data for deep CNNs in re-identification , 2017, Comput. Vis. Image Underst..

[67]  Lamiaa A. Elrefaei,et al.  A Survey on Deep Learning-Based Person Re-Identification Systems , 2019, IEEE Access.

[68]  Qijun Zhao,et al.  JL-DCF: Joint Learning and Densely-Cooperative Fusion Framework for RGB-D Salient Object Detection , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[69]  Fan Zhang,et al.  Distance learning by mining hard and easy negative samples for person re-identification , 2019, Pattern Recognit..