Robust Depth-Based Person Re-Identification

Person re-identification (re-id) aims to match people across non-overlapping camera views. So far the RGB-based appearance is widely used in most existing works. However, when people appeared in extreme illumination or changed clothes, the RGB appearance-based re-id methods tended to fail. To overcome this problem, we propose to exploit depth information to provide more invariant body shape and skeleton information regardless of illumination and color change. More specifically, we exploit depth voxel covariance descriptor and further propose a locally rotation invariant depth shape descriptor called Eigen-depth feature to describe pedestrian body shape. We prove that the distance between any two covariance matrices on the Riemannian manifold is equivalent to the Euclidean distance between the corresponding Eigen-depth features. Furthermore, we propose a kernelized implicit feature transfer scheme to estimate Eigen-depth feature implicitly from RGB image when depth information is not available. We find that combining the estimated depth features with RGB-based appearance features can sometimes help to better reduce visual ambiguities of appearance features caused by illumination and similar clothes. The effectiveness of our models was validated on publicly available depth pedestrian datasets as compared to related methods for re-id.

[1]  J. Wade Davis,et al.  Statistical Pattern Recognition , 2003, Technometrics.

[2]  Zhen Li,et al.  Learning Locally-Adaptive Decision Functions for Person Verification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Zheng Wang,et al.  Zero-Shot Person Re-identification via Cross-View Consistency , 2016, IEEE Transactions on Multimedia.

[4]  Gwenn Englebienne,et al.  Person re-identification using height-based gait in colour depth camera , 2013, 2013 IEEE International Conference on Image Processing.

[5]  Thomas B. Moeslund,et al.  Multimodal person re-identification using RGB-D sensors and a transient identification database , 2013, 2013 International Workshop on Biometrics and Forensics (IWBF).

[6]  Shaogang Gong,et al.  Learning a Discriminative Null Space for Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Jian-Huang Lai,et al.  Depth-based person re-identification , 2015, 2015 3rd IAPR Asian Conference on Pattern Recognition (ACPR).

[8]  Xiaogang Wang,et al.  Person Re-identification by Salience Matching , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  Bir Bhanu,et al.  Person Reidentification With Reference Descriptor , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Javier Lorenzo-Navarro,et al.  An Study on Re-identification in RGB-D Imagery , 2012, IWAAL.

[11]  Matteo Munaro,et al.  A feature-based approach to people re-identification using skeleton keypoints , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Matthias Rauterberg,et al.  People identification for domestic non-overlapping RGB-D camera networks , 2014, 2014 IEEE 11th International Multi-Conference on Systems, Signals & Devices (SSD14).

[13]  Xiaogang Wang,et al.  Learning Deep Feature Representations with Domain Guided Dropout for Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Stan Sclaroff,et al.  Improved feature descriptors for 3D surface matching , 2007, SPIE Optics East.

[15]  Bernhard Schölkopf,et al.  A Generalized Representer Theorem , 2001, COLT/EuroCOLT.

[16]  Bir Bhanu,et al.  Person Re-Identification by Robust Canonical Correlation Analysis , 2015, IEEE Signal Processing Letters.

[17]  Fei Xiong,et al.  Person Re-Identification Using Kernel-Based Metric Learning Methods , 2014, ECCV.

[18]  Alfredo Gardel Vicente,et al.  Person Re-Identification Ranking Optimisation by Discriminant Context Information Analysis , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[19]  Antonio Albiol,et al.  3D descriptor for people re-identification , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[20]  Bingpeng Ma,et al.  Local Descriptors Encoded by Fisher Vectors for Person Re-identification , 2012, ECCV Workshops.

[21]  Shaogang Gong,et al.  Towards Open-World Person Re-Identification by One-Shot Group-Based Verification , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Xiang Li,et al.  Partial Person Re-Identification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[23]  Nanning Zheng,et al.  Similarity learning on an explicit polynomial kernel feature map for person re-identification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Xiaojing Chen,et al.  Sparse representation matching for person re-identification , 2016, Inf. Sci..

[25]  W. Förstner,et al.  A Metric for Covariance Matrices , 2003 .

[26]  Jian-Huang Lai,et al.  Person Re-Identification by Camera Correlation Aware Feature Augmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Vittorio Murino,et al.  Custom Pictorial Structures for Re-identification , 2011, BMVC.

[28]  Shengcai Liao,et al.  Person re-identification by Local Maximal Occurrence representation and metric learning , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Hai Tao,et al.  Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features , 2008, ECCV.

[30]  Larry S. Davis,et al.  Multi-Task Learning with Low Rank Attribute Embedding for Person Re-Identification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[31]  Fabio Roli,et al.  Real-time Appearance-based Person Re-identification Over Multiple KinectTM Cameras , 2013, VISAPP.

[32]  Luc Van Gool,et al.  One-Shot Person Re-identification with a Consumer Depth Camera , 2014, Person Re-Identification.

[33]  Anil K. Jain,et al.  ViSE: Visual Search Engine Using Multiple Networked Cameras , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[34]  Xiaogang Wang,et al.  Shape and Appearance Context Modeling , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[35]  Tao Xiang,et al.  Transferring a semantic representation for person re-identification and search , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Li Fei-Fei,et al.  Recurrent Attention Models for Depth-Based Person Identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[37]  Kilian Q. Weinberger,et al.  Distance Metric Learning for Large Margin Nearest Neighbor Classification , 2005, NIPS.

[38]  S. Gong,et al.  Person Re-Identification by Unsupervised \ell _1 ℓ 1 Graph Learning , 2016, ECCV.

[39]  Ying Zhang,et al.  Gabor-LBP Based Region Covariance Descriptor for Person Re-identification , 2011, 2011 Sixth International Conference on Image and Graphics.

[40]  XiangTao,et al.  Towards Open-World Person Re-Identification by One-Shot Group-Based Verification , 2016 .

[41]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[42]  Gang Wang,et al.  Gated Siamese Convolutional Neural Network Architecture for Human Re-identification , 2016, ECCV.

[43]  Shaogang Gong,et al.  Reidentification by Relative Distance Comparison , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Michael Jones,et al.  An improved deep learning architecture for person re-identification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Qiuping Xu Canonical correlation Analysis , 2014 .

[46]  Rita Cucchiara,et al.  3DPeS: 3D people dataset for surveillance and forensics , 2011, J-HGBU '11.

[47]  Mubarak Shah,et al.  Human identity recognition in aerial images , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[48]  Shaogang Gong,et al.  Person Re-Identification by Unsupervised \ell _1 ℓ 1 Graph Learning , 2016, ECCV.

[49]  Horst Bischof,et al.  Large scale metric learning from equivalence constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[50]  Shaogang Gong,et al.  Human-in-the-Loop Person Re-identification , 2016, ECCV.

[51]  Yang Li,et al.  Person Re-Identification with Discriminatively Trained Viewpoint Invariant Dictionaries , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[52]  Xiao-Yuan Jing,et al.  Super-resolution Person re-identification with semi-coupled low-rank discriminant dictionary learning , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[53]  Alberto Del Bimbo,et al.  Person Re-Identification by Iterative Re-Weighted Sparse Ranking , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[54]  Chunxiao Liu,et al.  Person Re-identification: What Features Are Important? , 2012, ECCV Workshops.

[55]  Alessio Del Bue,et al.  Re-identification with RGB-D Sensors , 2012, ECCV Workshops.

[56]  Jian-Huang Lai,et al.  An Asymmetric Distance Model for Cross-View Feature Mapping in Person Reidentification , 2017, IEEE Transactions on Circuits and Systems for Video Technology.

[57]  Shaogang Gong,et al.  Person Re-Identification by Support Vector Ranking , 2010, BMVC.

[58]  Erik W. Grafarend,et al.  Geodesy-The Challenge of the 3rd Millennium , 2003 .

[59]  Shaogang Gong,et al.  Unsupervised Cross-Dataset Transfer Learning for Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[60]  Frédéric Jurie,et al.  PCCA: A new approach for distance learning from sparse pairwise constraints , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[61]  Tony DeRose,et al.  Surface reconstruction from unorganized points , 1992, SIGGRAPH.

[62]  J. M. Mossi,et al.  Who is who at different cameras: people re-identification using depth cameras , 2012 .

[63]  Jianxin Wu,et al.  Person Re-Identification with Correspondence Structure Learning , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[64]  Sergio A. Velastin,et al.  Local Fisher Discriminant Analysis for Pedestrian Re-identification , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[65]  Alessandro Perina,et al.  Person re-identification by symmetry-driven accumulation of local features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[66]  Sergio Escalera,et al.  Tri-modal Person Re-identification with RGB, Depth and Thermal Features , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[67]  Shengcai Liao,et al.  Salient Color Names for Person Re-identification , 2014, ECCV.

[68]  Xiang Li,et al.  An enhanced deep feature representation for person re-identification , 2016, 2016 IEEE Winter Conference on Applications of Computer Vision (WACV).

[69]  Fabio Roli,et al.  Multimodal Person Reidentification Using RGB-D Cameras , 2016, IEEE Transactions on Circuits and Systems for Video Technology.

[70]  Shengcai Liao,et al.  Efficient PSD Constrained Asymmetric Metric Learning for Person Re-Identification , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[71]  Slawomir Bak,et al.  Person Re-identification Using Haar-based and DCD-based Signature , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[72]  Department of Electrical,et al.  Computational and Performance Aspects of PCA-Based Face-Recognition Algorithms , 2001, Perception.

[73]  Dong Xu,et al.  Distance Metric Learning Using Privileged Information for Face Verification and Person Re-Identification , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[74]  Luc Van Gool,et al.  3D reconstruction of freely moving persons for re-identification with a depth sensor , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[75]  Rita Cucchiara,et al.  Learning articulated body models for people re-identification , 2013, MM '13.

[76]  Xiaogang Wang,et al.  Learning Mid-level Filters for Person Re-identification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[77]  Shiliang Zhang,et al.  Deep Attributes Driven Multi-Camera Person Re-identification , 2016, ECCV.

[78]  Jian-Huang Lai,et al.  Mirror Representation for Modeling View-Specific Transform in Person Re-Identification , 2015, IJCAI.

[79]  Xiang Li,et al.  Top-Push Video-Based Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[80]  Javier Lorenzo-Navarro,et al.  People Semantic Description and Re-identification from Point Cloud Geometry , 2014, 2014 22nd International Conference on Pattern Recognition.

[81]  Xiaogang Wang,et al.  DeepReID: Deep Filter Pairing Neural Network for Person Re-identification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[82]  Vassilios Morellas,et al.  Compact covariance descriptors in 3D point clouds for object recognition , 2012, 2012 IEEE International Conference on Robotics and Automation.

[83]  Dacheng Tao,et al.  Person Re-Identification Over Camera Networks Using Multi-Task Distance Metric Learning , 2014, IEEE Transactions on Image Processing.

[84]  Nikolaos Papanikolopoulos,et al.  RGB-D object classification using covariance descriptors , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[85]  Slawomir Bak,et al.  Re-identification by Covariance Descriptors , 2014, Person Re-Identification.

[86]  Anton van den Hengel,et al.  Learning to rank in person re-identification with metric ensembles , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[87]  Larry S. Davis,et al.  Learning Discriminative Appearance-Based Models Using Partial Least Squares , 2009, 2009 XXII Brazilian Symposium on Computer Graphics and Image Processing.

[88]  Ehud Rivlin,et al.  Color Invariants for Person Reidentification , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.