Three-dimensional convolutional neural networks applied to video sensor-based gait recognition

In this paper, we propose a novel gait representation based on 3D-CNN, i.e., learning spatio-temporal multi-scale gait identity features (GaitID) using the 3-dimensional convolutional networks. Our contributions include: 1) explore different numbers of input frames for 3D-CNN model, 2) evaluate different features and gait representations in 3D-CNN, and 3) improve the net structure to learn multi-scale gait features with low dimensions. Nearest neighbor (NN) classifier was applied to identify the gait. When compared with other existing methods, the results reported on the CASIA-B dataset demonstrated that the proposed method not only achieved a competitive performance, but also still retained the discriminative power in a very low dimension (128-D), even with a simpler classifier.

[1]  Xiaogang Wang,et al.  Deeply learned face representations are sparse, selective, and robust , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Cordelia Schmid,et al.  Aggregating local descriptors into a compact image representation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[3]  Reza Safabakhsh,et al.  Model-based human gait recognition using leg and arm movements , 2010, Eng. Appl. Artif. Intell..

[4]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[5]  Rama Chellappa,et al.  Towards a view invariant gait recognition algorithm , 2003, Proceedings of the IEEE Conference on Advanced Video and Signal Based Surveillance, 2003..

[6]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[7]  Yasushi Makihara,et al.  The OU-ISIR Gait Database Comprising the Large Population Dataset and Performance Evaluation of Gait Recognition , 2012, IEEE Transactions on Information Forensics and Security.

[8]  Thomas Mensink,et al.  Image Classification with the Fisher Vector: Theory and Practice , 2013, International Journal of Computer Vision.

[9]  Yoshua Bengio,et al.  Scaling learning algorithms towards AI , 2007 .

[10]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[11]  Xiaogang Wang,et al.  Deep Learning Face Representation by Joint Identification-Verification , 2014, NIPS.

[12]  Tao Xiang,et al.  Gait Recognition by Ranking , 2012, ECCV.

[13]  Yann LeCun,et al.  Traffic sign recognition with multi-scale Convolutional Networks , 2011, The 2011 International Joint Conference on Neural Networks.

[14]  Yasushi Makihara,et al.  Clothing-invariant gait identification using part-based clothing categorization and adaptive weight control , 2010, Pattern Recognit..

[15]  Shaogang Gong,et al.  Gait recognition using Gait Entropy Image , 2009, ICDP.

[16]  James J. Little,et al.  Incremental Learning for Video-Based Gait Recognition With LBP Flow , 2013, IEEE Transactions on Cybernetics.

[17]  Thomas Wolf,et al.  Multi-view gait recognition using 3D convolutional neural networks , 2016, 2016 IEEE International Conference on Image Processing (ICIP).

[18]  Xiaogang Wang,et al.  A Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Thomas B. Moeslund,et al.  Invariant gait continuum based on the duty-factor , 2009, Signal Image Video Process..

[21]  Yasushi Makihara,et al.  Gait-Based Person Recognition Using Arbitrary View Transformation Model , 2015, IEEE Transactions on Image Processing.

[22]  Lorenzo Torresani,et al.  Learning Spatiotemporal Features with 3D Convolutional Networks , 2014, 2015 IEEE International Conference on Computer Vision (ICCV).

[23]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[24]  Saeid Nahavandi,et al.  A Review of Vision-Based Gait Recognition Methods for Human Identification , 2010, 2010 International Conference on Digital Image Computing: Techniques and Applications.

[25]  Shaogang Gong,et al.  Feature Selection for Gait Recognition without Subject Cooperation , 2008, BMVC.

[26]  Wei Xiong,et al.  Active energy image plus 2DLPP for gait recognition , 2010, Signal Process..

[27]  Yasushi Makihara,et al.  GEINet: View-invariant gait recognition using a convolutional neural network , 2016, 2016 International Conference on Biometrics (ICB).

[28]  Smriti Srivastava,et al.  Gait based authentication using gait information image features , 2015, Pattern Recognit. Lett..

[29]  Qiang Wu,et al.  Gait Recognition Under Various Viewing Angles Based on Correlated Motion Regression , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[30]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[31]  Yasushi Makihara,et al.  Gait Recognition Using a View Transformation Model in the Frequency Domain , 2006, ECCV.

[32]  Thomas Mensink,et al.  Improving the Fisher Kernel for Large-Scale Image Classification , 2010, ECCV.

[33]  Yi Yang,et al.  A discriminative CNN video representation for event detection , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Worapan Kusakunniran,et al.  Recognizing Gaits on Spatio-Temporal Feature Domain , 2014, IEEE Transactions on Information Forensics and Security.

[35]  Mark S. Nixon,et al.  Automated person recognition by walking and running via model-based approaches , 2004, Pattern Recognit..

[36]  Chao Li,et al.  DeepGait: A Learning Deep Convolutional Representation for View-Invariant Gait Recognition Using Joint Bayesian , 2017 .

[37]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[38]  Xiang Li,et al.  Gait Energy Response Function for Clothing-Invariant Gait Recognition , 2016, ACCV.

[39]  Shaogang Gong,et al.  Gait recognition without subject cooperation , 2010, Pattern Recognit. Lett..

[40]  Arun Ross,et al.  Gait curves for human recognition, backpack detection, and silhouette correction in a nighttime environment , 2010, Defense + Commercial Sensing.

[41]  Cordelia Schmid,et al.  Aggregating Local Image Descriptors into Compact Codes , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Neha Jain,et al.  Gait recognition based on gait pal and pal entropy image , 2013, 2013 IEEE International Conference on Image Processing.

[43]  Tieniu Tan,et al.  A Framework for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[44]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[45]  Xianglei Xing,et al.  Complete canonical correlation analysis with application to multi-view gait recognition , 2016, Pattern Recognit..

[46]  Yasushi Makihara,et al.  Cross-view gait recognition using view-dependent discriminative analysis , 2014, IEEE International Joint Conference on Biometrics.

[47]  Sudeep Sarkar,et al.  The humanID gait challenge problem: data sets, performance, and analysis , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Edward H. Adelson,et al.  Analyzing and recognizing walking figures in XYT , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.