Dynamic long short-term memory network for skeleton-based gait recognition

As an important biometric, gait has been extensively used for human recognition. Currently gait analysis based on 3D skeleton data are more commonly studied due to its accuracy, succinctness, and view-invariant representation. Traditional methods generally extract geometric features of human body and temporal dynamics characteristic. However, designing features from ad-hoc schemes mainly is based on experience, and thus it is difficult to obtain high-dimensional features beyond the limits of human interpretability. In this paper, considering that Long Short-Term Memory(LSTM) recurrent neural network can model the long-term contextual information of time sequences well, we first propose a dynamic approach to LSTM network for gait recognition from skeleton data. These skeleton data is essentially the discriminative information without extracting any hand-crafted features in advance. The framework is able to process sequences with variable length. Our experiments based on three publicly available datasets demonstrate that the proposed gait recognition method is effective and outperformed those in literature.

[1]  Rama Chellappa,et al.  Gait Analysis for Human Identification , 2003, AVBPA.

[2]  Azhin Tahir Sabir,et al.  Gait recognition based on Kinect sensor , 2014, Photonics Europe.

[3]  Li Fei-Fei,et al.  Recurrent Attention Models for Depth-Based Person Identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Liu Ming,et al.  A Wearable Acceleration Sensor System for Gait Recognition , 2007, 2007 2nd IEEE Conference on Industrial Electronics and Applications.

[5]  Alex Graves,et al.  Grid Long Short-Term Memory , 2015, ICLR.

[6]  Einar Snekkenes,et al.  Gait Recognition Using Wearable Motion Recording Sensors , 2009, EURASIP J. Adv. Signal Process..

[7]  Andrzej Polanski,et al.  Dynamic Time Warping in Gait Classificationof Motion Capture Data , 2012 .

[8]  Rama Chellappa,et al.  A hidden Markov model based framework for recognition of humans from gait sequences , 2003, Proceedings 2003 International Conference on Image Processing (Cat. No.03CH37429).

[9]  Fei Zhang,et al.  Relative distance features for gait recognition with Kinect , 2016, Journal of Visual Communication and Image Representation.

[10]  Gang Wang,et al.  Spatio-Temporal LSTM with Trust Gates for 3D Human Action Recognition , 2016, ECCV.

[11]  Robert T. Collins,et al.  Silhouette-based human identification from body shape and gait , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[12]  Christoph Busch,et al.  Unobtrusive User-Authentication on Mobile Phones Using Biometric Gait Recognition , 2010, 2010 Sixth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[13]  Jesús Martínez del Rincón,et al.  Recurrent Convolutional Network for Video-Based Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Gang Wang,et al.  NTU RGB+D: A Large Scale Dataset for 3D Human Activity Analysis , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Tieniu Tan,et al.  Silhouette Analysis-Based Gait Recognition for Human Identification , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[16]  Hai Tao,et al.  Viewpoint Invariant Pedestrian Recognition with an Ensemble of Localized Features , 2008, ECCV.

[17]  Matteo Gadaleta,et al.  IDNet: Smartphone-based Gait Recognition with Convolutional Neural Networks , 2016, Pattern Recognit..

[18]  Yasushi Makihara,et al.  Gait Verification System for Criminal Investigation , 2013, IPSJ Trans. Comput. Vis. Appl..

[19]  Ricardo Matsumura de Araújo,et al.  Person Identification Using Anthropometric and Gait Data from Kinect Sensor , 2015, AAAI.

[20]  Adam Świtoński,et al.  Dynamic Time Warping In Gait Classification of Motion Capture Data , .

[21]  Ehud Rivlin,et al.  Color Invariants for Person Reidentification , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Marco Grangetto,et al.  Human Classification Using Gait Features , 2014, BIOMET.

[23]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[24]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[25]  Petr Sojka,et al.  Learning robust features for gait recognition by Maximum Margin Criterion , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[26]  Jay Dicharry,et al.  Kinematics and kinetics of gait: from lab to clinic. , 2010, Clinics in sports medicine.

[27]  Stacy J. Morris,et al.  A shoe-integrated sensor system for wireless gait analysis and real-time therapeutic feedback , 2004 .

[28]  Tao Liu,et al.  Gait Analysis Using Wearable Sensors , 2012, Sensors.

[29]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[30]  Marina L. Gavrilova,et al.  DTW-based kernel and rank-level fusion for 3D gait recognition using Kinect , 2015, The Visual Computer.