Multi-Task GANs for View-Specific Feature Learning in Gait Recognition

Gait recognition is of great importance in the fields of surveillance and forensics to identify human beings since gait is the unique biometric feature that can be perceived efficiently at a distance. However, the accuracy of gait recognition to some extent suffers from both the variation of view angles and the deficient gait templates. On one hand, the existing cross-view methods focus on transforming gait templates among different views, which may accumulate the transformation error in a large variation of view angles. On the other hand, a commonly used gait energy image template loses temporal information of a gait sequence. To address these problems, this paper proposes multi-task generative adversarial networks (MGANs) for learning view-specific feature representations. In order to preserve more temporal information, we also propose a new multi-channel gait template, called period energy image (PEI). Based on the assumption of view angle manifold, the MGANs can leverage adversarial training to extract more discriminative features from gait sequences. Experiments on OU-ISIR, CASIA-B, and USF benchmark data sets indicate that compared with several recently published approaches, PEI + MGANs achieves competitive performance and is more interpretable to cross-view gait recognition.

[1]  Xiang Li,et al.  Joint Intensity and Spatial Metric Learning for Robust Gait Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Colin Fyfe,et al.  Kernel and Nonlinear Canonical Correlation Analysis , 2000, IJCNN.

[3]  James J. Little,et al.  View-Invariant Discriminative Projection for Multi-View Gait-Based Human Identification , 2013, IEEE Transactions on Information Forensics and Security.

[4]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  拓海 杉山,et al.  “Unpaired Image-to-Image Translation using Cycle-Consistent Adversarial Networks”の学習報告 , 2017 .

[6]  Gunawan Ariyanto,et al.  Model-based 3D gait biometrics , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[7]  Simon Osindero,et al.  Conditional Generative Adversarial Nets , 2014, ArXiv.

[8]  Xiaogang Wang,et al.  A Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Amit K. Roy-Chowdhury,et al.  A study on view-insensitive gait recognition , 2005, IEEE International Conference on Image Processing 2005.

[10]  Soumith Chintala,et al.  Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks , 2015, ICLR.

[11]  Yasushi Makihara,et al.  On Input/Output Architectures for Convolutional Neural Network-Based Cross-View Gait Recognition , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[12]  Arun Ross,et al.  Biometric recognition by gait: A survey of modalities and features , 2018, Comput. Vis. Image Underst..

[13]  Shaogang Gong,et al.  Cross View Gait Recognition Using Correlation Strength , 2010, BMVC.

[14]  Jesús Martínez del Rincón,et al.  Recurrent Convolutional Network for Video-Based Person Re-identification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Yang Feng,et al.  Learning effective Gait features using LSTM , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[16]  Yasushi Makihara,et al.  The OU-ISIR Gait Database Comprising the Large Population Dataset and Performance Evaluation of Gait Recognition , 2012, IEEE Transactions on Information Forensics and Security.

[17]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[18]  Mark S. Nixon,et al.  Self-Calibrating View-Invariant Gait Biometrics , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[19]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[20]  Shiqi Yu,et al.  GaitGAN: Invariant Gait Feature Extraction Using Generative Adversarial Networks , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[21]  Smriti Srivastava,et al.  Gait recognition using gait Gaussian image , 2015, 2015 2nd International Conference on Signal Processing and Integrated Networks (SPIN).

[22]  Rudolf Fleischer,et al.  Low-Resolution Gait Recognition , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[23]  James Nga-Kwok Liu,et al.  Gait flow image: A silhouette-based gait representation for human identification , 2011, Pattern Recognit..

[24]  Christian Ledig,et al.  Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[25]  Qiang Wu,et al.  Recognizing Gaits Across Views Through Correlated Motion Co-Clustering , 2014, IEEE Transactions on Image Processing.

[26]  Qiang Wu,et al.  A New View-Invariant Feature for Cross-View Gait Recognition , 2013, IEEE Transactions on Information Forensics and Security.

[27]  Tieniu Tan,et al.  A Framework for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[28]  Jean-Luc Dugelay,et al.  Face aging with conditional generative adversarial networks , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[29]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[30]  Hua Li,et al.  3D gait recognition using multiple cameras , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[31]  Chen Wang,et al.  Chrono-Gait Image: A Novel Temporal Template for Gait Recognition , 2010, ECCV.

[32]  Osama Masoud,et al.  View-independent human motion classification using image-based reconstruction , 2009, Image Vis. Comput..

[33]  Chen Wang,et al.  Human Identification Using Temporal Information Preserving Gait Template , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Haifeng Hu,et al.  Enhanced Gabor Feature Based Classification Using a Regularized Locally Tensor Discriminant Model for Multiview Gait Recognition , 2013, IEEE Transactions on Circuits and Systems for Video Technology.

[35]  Can Luo,et al.  Robust gait recognition based on partitioning and canonical correlation analysis , 2015, 2015 IEEE International Conference on Imaging Systems and Techniques (IST).

[36]  Sudeep Sarkar,et al.  The humanID gait challenge problem: data sets, performance, and analysis , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Xianglei Xing,et al.  Complete canonical correlation analysis with application to multi-view gait recognition , 2016, Pattern Recognit..

[38]  Qiang Wu,et al.  Support vector regression for multi-view gait recognition based on local motion feature selection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[39]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[40]  LinLin Shen,et al.  Invariant feature extraction for gait recognition using only one uniform model , 2017, Neurocomputing.

[41]  Yasushi Makihara,et al.  Gait Recognition Using a View Transformation Model in the Frequency Domain , 2006, ECCV.

[42]  Minh N. Do,et al.  Semantic Image Inpainting with Deep Generative Models , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[43]  Qiang Wu,et al.  Multiple views gait recognition using View Transformation Model based on optimized Gait Energy Image , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[44]  Peter Peer,et al.  Frame–based classification for cross-speed gait recognition , 2017, Multimedia Tools and Applications.

[45]  Shiqi Yu,et al.  Pose-Based Temporal-Spatial Network (PTSN) for Gait Recognition with Carrying and Clothing Variations , 2017, CCBR.

[46]  Robert Bergevin,et al.  Computing and evaluating view-normalized body part trajectories , 2009, Image Vis. Comput..

[47]  Yasushi Makihara,et al.  GEINet: View-invariant gait recognition using a convolutional neural network , 2016, 2016 International Conference on Biometrics (ICB).

[48]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.