Tri-modal Person Re-identification with RGB, Depth and Thermal Features

Person re-identification is about recognizing people who have passed by a sensor earlier. Previous work is mainly based on RGB data, but in this work we for the first time present a system where we combine RGB, depth, and thermal data for re-identification purposes. First, from each of the three modalities, we obtain some particular features: from RGB data, we model color information from different regions of the body, from depth data, we compute different soft body biometrics, and from thermal data, we extract local structural information. Then, the three information types are combined in a joined classifier. The tri-modal system is evaluated on a new RGB-D-T dataset, showing successful results in re-identification scenarios.

[1]  H. Opower.  Multiple view geometry in computer vision , 2002 .

[2]  Michael Arens,et al.  A Multi-staged System for Efficient Visual Person Reidentification , 2011, MVA.

[3]  Alessandro Perina,et al.  Person re-identification by symmetry-driven accumulation of local features , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[4]  J R Dumouchel,et al.  Urban planning. , 1969, Hospitals.

[5]  Horst Bischof,et al.  Person Re-identification by Descriptive and Discriminative Classification , 2011, SCIA.

[6]  Slawomir Bak,et al.  Learning to Match Appearances by Correlations in a Covariance Metric Space , 2012, ECCV.

[7]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[8]  Marko Robnik-Sikonja,et al.  Theoretical and Empirical Analysis of ReliefF and RReliefF , 2003, Machine Learning.

[9]  Mohan M. Trivedi,et al.  Mutual information based registration of multimodal stereo videos for person tracking , 2007, Comput. Vis. Image Underst..

[10]  Mubarak Shah,et al.  Identifying Behaviors in Crowd Scenes Using Stability Analysis for Dynamical Systems , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Sadiye Guler,et al.  Automated person categorization for video surveillance using soft biometrics , 2010, Defense + Commercial Sensing.

[12]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[13]  Shaogang Gong,et al.  International Journal of Computer Vision (The original publication is available at www.springerlink.com) Time-Delayed Correlation Analysis for Multi-Camera Activity Understanding , 2009 .

[14]  Peter H. Tu,et al.  Appearance-based person reidentification in camera networks: problem overview and current approaches , 2011, J. Ambient Intell. Humaniz. Comput..

[15]  Michael Arens,et al.  Local Feature Based Person Reidentification in Infrared Image Sequences , 2010, 2010 7th IEEE International Conference on Advanced Video and Signal Based Surveillance.

[16]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[17]  Mubarak Shah,et al.  Visual crowd surveillance through a hydrodynamics lens , 2011, Commun. ACM.

[18]  Sridha Sridharan,et al.  A Mask-Based Approach for the Geometric Calibration of Thermal-Infrared Cameras , 2012, IEEE Transactions on Instrumentation and Measurement.

[19]  Shaogang Gong,et al.  Person re-identification by probabilistic relative distance comparison , 2011, CVPR 2011.

[20]  Jean-Luc Dugelay,et al.  Improving identification by pruning: A case study on face recognition and body soft biometric , 2012, 2012 13th International Workshop on Image Analysis for Multimedia Interactive Services.

[21]  Alessio Del Bue,et al.  Re-identification with RGB-D Sensors , 2012, ECCV Workshops.

[22]  Teri A. Crosby,et al.  How to Detect and Handle Outliers , 1993 .