Spatio-temporal metric learning for individual recognition from locomotion

Abstract Individual recognition from locomotion is a challenging task owing to large intra-class and small inter-class variations. In this article, we present a novel metric learning method for individual recognition from skeleton sequences. Firstly, we propose to model articulated body on Riemannian manifold to describe the essence of human motion, which can reflect biometric signatures of the enrolled individuals. Then two spatia-temporal metric learning approaches are proposed, namely Spatio-Temporal Large Margin Nearest Neighbor (ST-LMNN) and Spatio-Temporal Multi-Metric Learning (STMM), to learn discriminant bilinear metrics which can encode the spatio-temporal structure of human motion. Specifically, the ST-LMNN algorithm extends the bilinear model into classical Large Margin Nearest Neighbor method, which learns a low-dimensional local linear embedding in the spatial and temporal domain, respectively. To further capture the unique motion pattern for each individual, the proposed STMM algorithm learns a set of individual-specific spatio-temporal metrics, which make the projected features of the same person closer to its class mean than that of different classes by a large margin. Beyond that, we present a new publicly available dataset for locomotion recognition to evaluate the influence of both internal and external covariant factors. According to the experimental results from the three public datasets, we believe that the proposed approaches are both able to achieve competitive results in individual recognition.