Enhanced skeleton and face 3D data for person re-identification from depth cameras

Abstract Person re-identification is typically performed using 2D still images or videos, where photometric appearance is the main visual cue used to discover the presence of a target subject when switching from different camera views across time. This invalidates any application where a person may change dress across subsequent acquisitions as can be the case of patients monitoring at home. Differently from RGB data, 3D information as acquired by depth cameras can open the way to person re-identification based on biometric cues such as distinguishing traits of the body or face. However, the accuracy of skeleton and face geometry extracted from depth data is not always adequate to enable person recognition, since both these features are affected by the pose of the subject and the distance from the camera. In this paper, we propose a method to derive a robust skeleton representation from a depth sequence and to complement it with a highly discriminative face feature. This is obtained by selecting skeleton and face samples based on their quality and using the temporal redundancy across the sequence to derive and refine cumulated models for both of them. Extracting skeleton and face features from such cumulated models and combining them for the recognition allow us to improve rank-1 re-identification accuracy compared to individual cues. A comparative evaluation on three benchmark datasets also shows results at the state-of-the-art.