A Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs

This paper studies an approach to gait based human identification via similarity learning by deep convolutional neural networks (CNNs). With a pretty small group of labeled multi-view human walking videos, we can train deep networks to recognize the most discriminative changes of gait patterns which suggest the change of human identity. To the best of our knowledge, this is the first work based on deep CNNs for gait recognition in the literature. Here, we provide an extensive empirical evaluation in terms of various scenarios, namely, cross-view and cross-walking-condition, with different preprocessing approaches and network architectures. The method is first evaluated on the challenging CASIA-B dataset in terms of cross-view gait recognition. Experimental results show that it outperforms the previous state-of-the-art methods by a significant margin. In particular, our method shows advantages when the cross-view angle is large, i.e., no less than 36 degree. And the average recognition rate can reach 94 percent, much better than the previous best result (less than 65 percent). The method is further evaluated on the OU-ISIR gait dataset to test its generalization ability to larger data. OU-ISIR is currently the largest dataset available in the literature for gait recognition, with 4,007 subjects. On this dataset, the average accuracy of our method under identical view conditions is above 98 percent, and the one for cross-view scenarios is above 91 percent. Finally, the method also performs the best on the USF gait dataset, whose gait sequences are imaged in a real outdoor scene. These results show great potential of this method for practical applications.

[1]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[2]  Zhenhua Guo,et al.  Face recognition by sparse discriminant analysis via joint L2, 1-norm minimization , 2014, Pattern Recognit..

[3]  Gunawan Ariyanto,et al.  Model-based 3D gait biometrics , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[4]  Ali Farhadi,et al.  Learning to Recognize Activities from the Wrong View Point , 2008, ECCV.

[5]  Stefano Soatto,et al.  Hybrid Dynamical Models of Human Motion for the Recognition of Human Gaits , 2009, International Journal of Computer Vision.

[6]  Fei-Fei Li,et al.  Large-Scale Video Classification with Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  David Zhang,et al.  Study on novel Curvature Features for 3D fingerprint recognition , 2015, Neurocomputing.

[8]  Cordelia Schmid,et al.  Action Recognition with Improved Trajectories , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[11]  Christian Szegedy,et al.  DeepPose: Human Pose Estimation via Deep Neural Networks , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Tao Xiang,et al.  Uncooperative gait recognition by learning to rank , 2014, Pattern Recognit..

[13]  Qiang Wu,et al.  Gait Recognition Under Various Viewing Angles Based on Correlated Motion Regression , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[14]  Junjie Yan,et al.  Convolutional Channel Features: Tailoring CNN to Diverse Tasks , 2015 .

[15]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[16]  Rémi Ronfard,et al.  Free viewpoint action recognition using motion history volumes , 2006, Comput. Vis. Image Underst..

[17]  Imed Bouchrika,et al.  On Using Gait in Forensic Biometrics , 2011, Journal of forensic sciences.

[18]  Yasushi Makihara,et al.  The OU-ISIR Gait Database Comprising the Large Population Dataset and Performance Evaluation of Gait Recognition , 2012, IEEE Transactions on Information Forensics and Security.

[19]  Sridha Sridharan,et al.  A Database for Person Re-Identification in Multi-Camera Surveillance Networks , 2012, 2012 International Conference on Digital Image Computing Techniques and Applications (DICTA).

[20]  Mark S. Nixon,et al.  Self-Calibrating View-Invariant Gait Biometrics , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[21]  Yann LeCun,et al.  Learning a similarity metric discriminatively, with application to face verification , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[22]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[23]  Qiang Wu,et al.  Multiple views gait recognition using View Transformation Model based on optimized Gait Energy Image , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[24]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[25]  Guo-Jun Qi,et al.  Differential Recurrent Neural Networks for Action Recognition , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[26]  Yann LeCun,et al.  Convolutional networks and applications in vision , 2010, Proceedings of 2010 IEEE International Symposium on Circuits and Systems.

[27]  Qiang Wu,et al.  Support vector regression for multi-view gait recognition based on local motion feature selection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[28]  Cordelia Schmid,et al.  Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Categories , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[29]  Yasushi Makihara,et al.  Gait Recognition Using a View Transformation Model in the Frequency Domain , 2006, ECCV.

[30]  James A. Reggia,et al.  Robust human action recognition via long short-term memory , 2013, The 2013 International Joint Conference on Neural Networks (IJCNN).

[31]  Lawrence D. Jackel,et al.  Handwritten Digit Recognition with a Back-Propagation Network , 1989, NIPS.

[32]  Tieniu Tan,et al.  Early Hierarchical Contexts Learned by Convolutional Networks for Image Segmentation , 2014, 2014 22nd International Conference on Pattern Recognition.

[33]  Andrew Zisserman,et al.  Two-Stream Convolutional Networks for Action Recognition in Videos , 2014, NIPS.

[34]  Haifeng Hu,et al.  Enhanced Gabor Feature Based Classification Using a Regularized Locally Tensor Discriminant Model for Multiview Gait Recognition , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  Chen Wang,et al.  Chrono-Gait Image: A Novel Temporal Template for Gait Recognition , 2010, ECCV.

[36]  Osama Masoud,et al.  View-independent human motion classification using image-based reconstruction , 2009, Image Vis. Comput..

[37]  Marwan Mattar,et al.  Labeled Faces in the Wild: A Database forStudying Face Recognition in Unconstrained Environments , 2008 .

[38]  Qiang Wu,et al.  A New View-Invariant Feature for Cross-View Gait Recognition , 2013, IEEE Transactions on Information Forensics and Security.

[39]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[40]  Tieniu Tan,et al.  A Framework for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[41]  Ming Yang,et al.  3D Convolutional Neural Networks for Human Action Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Qiang Wu,et al.  Recognizing Gaits Across Views Through Correlated Motion Co-Clustering , 2014, IEEE Transactions on Image Processing.

[43]  Xiang Zhang,et al.  OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks , 2013, ICLR.

[44]  James Nga-Kwok Liu,et al.  Gait flow image: A silhouette-based gait representation for human identification , 2011, Pattern Recognit..

[45]  Hua Li,et al.  3D gait recognition using multiple cameras , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[46]  Bin Yang,et al.  Convolutional Channel Features , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[47]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Tieniu Tan,et al.  Toward Accurate and Fast Iris Segmentation for Iris Biometrics , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[49]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[50]  Xiaogang Wang,et al.  Hybrid Deep Learning for Face Verification , 2013, 2013 IEEE International Conference on Computer Vision.

[51]  Sudeep Sarkar,et al.  The humanID gait challenge problem: data sets, performance, and analysis , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  David Zhang,et al.  Human Gait Recognition via Sparse Discriminant Projection Learning , 2014, IEEE Transactions on Circuits and Systems for Video Technology.

[53]  James J. Little,et al.  View-Invariant Discriminative Projection for Multi-View Gait-Based Human Identification , 2013, IEEE Transactions on Information Forensics and Security.

[54]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Shaogang Gong,et al.  Cross View Gait Recognition Using Correlation Strength , 2010, BMVC.

[57]  Trevor Darrell,et al.  Fully Convolutional Networks for Semantic Segmentation , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.