A general tensor representation framework for cross-view gait recognition

Abstract Tensor analysis methods have played an important role in identifying human gaits using high dimensional data. However, when view angles change, it becomes more and more difficult to recognize cross-view gait by learning only a set of multi-linear projection matrices. To address this problem, a general tensor representation framework for cross-view gait recognition is proposed in this paper. There are three criteria of tensorial coupled mappings in the proposed framework. (1) Coupled multi-linear locality-preserved criterion (CMLP) aims to detect the essential tensorial manifold structure via preserving local information. (2) Coupled multi-linear marginal fisher criterion (CMMF) aims to encode the intra-class compactness and inter-class separability with local relationships. (3) Coupled multi-linear discriminant analysis criterion (CMDA) aims to minimize the intra-class scatter and maximize the inter-class scatter. For the three tensor algorithms for cross-view gaits, two sets of multi-linear projection matrices are iteratively learned using alternating projection optimization procedures. The proposed methods are compared with the recently published cross-view gait recognition approaches on CASIA(B) and OU-ISIR gait database. The results demonstrate that the performances of the proposed methods are superior to existing state-of-the-art cross-view gait recognition approaches.

[1]  Tieniu Tan,et al.  Modelling the Effect of View Angle Variation on Appearance-Based Gait Recognition , 2006, ACCV.

[2]  Peng Zhang,et al.  On the distance metric learning between cross-domain gaits , 2016, Neurocomputing.

[3]  Xianglei Xing,et al.  Complete canonical correlation analysis with application to multi-view gait recognition , 2016, Pattern Recognit..

[4]  Qiang Wu,et al.  Support vector regression for multi-view gait recognition based on local motion feature selection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[5]  Yasushi Makihara,et al.  Gait Identification Based on Multi-view Observations Using Omnidirectional Camera , 2007, ACCV.

[6]  Xiaodong Yu,et al.  Learning Bidirectional Temporal Cues for Video-Based Person Re-Identification , 2018, IEEE Transactions on Circuits and Systems for Video Technology.

[7]  Chiou-Ting Hsu,et al.  Multilinear Graph Embedding: Representation and Regularization for Images , 2014, IEEE Transactions on Image Processing.

[8]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[9]  Chang-Tsun Li,et al.  On Reducing the Effect of Covariate Factors in Gait Recognition: A Classifier Ensemble Method , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Gunawan Ariyanto,et al.  Model-based 3D gait biometrics , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[11]  Jiwen Lu,et al.  Joint Subspace Learning for View-Invariant Gait Recognition , 2011, IEEE Signal Processing Letters.

[12]  Qiang Wu,et al.  Multi-view Gait Recognition Based on Motion Regression Using Multilayer Perceptron , 2010, 2010 20th International Conference on Pattern Recognition.

[13]  Sudeep Sarkar,et al.  Statistical Motion Model Based on the Change of Feature Relationships: Human Gait-Based Recognition , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[14]  Xuelong Li,et al.  Discriminant Locally Linear Embedding With High-Order Tensor Data , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[15]  Ya Zhang,et al.  Part-Stacked CNN for Fine-Grained Visual Categorization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Yasushi Makihara,et al.  GEINet: View-invariant gait recognition using a convolutional neural network , 2016, 2016 International Conference on Biometrics (ICB).

[17]  Yasushi Makihara,et al.  Which Reference View is Effective for Gait Identification Using a View Transformation Model? , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[18]  Xuelong Li,et al.  General Tensor Discriminant Analysis and Gabor Features for Gait Recognition , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Tardi Tjahjadi,et al.  Robust arbitrary view gait recognition based on parametric 3D human body reconstruction and virtual posture synthesis , 2016, Pattern Recognit..

[20]  Tieniu Tan,et al.  A Framework for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[21]  Dacheng Tao,et al.  Trunk-Branch Ensemble Convolutional Neural Networks for Video-Based Face Recognition , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Wei Xiong,et al.  Stacked Convolutional Denoising Auto-Encoders for Feature Representation , 2017, IEEE Transactions on Cybernetics.

[23]  Yasushi Makihara,et al.  The OU-ISIR Gait Database Comprising the Large Population Dataset and Performance Evaluation of Gait Recognition , 2012, IEEE Transactions on Information Forensics and Security.

[24]  Mark S. Nixon,et al.  Self-Calibrating View-Invariant Gait Biometrics , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[25]  Rui Yan,et al.  Dual-ellipse fitting approach for robust gait periodicity detection , 2012, Neurocomputing.

[26]  Chen Wang,et al.  Human Identification Using Temporal Information Preserving Gait Template , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Tieniu Tan,et al.  Robust view transformation model for gait recognition , 2011, 2011 18th IEEE International Conference on Image Processing.

[28]  Qiang Wu,et al.  Gait Recognition Under Various Viewing Angles Based on Correlated Motion Regression , 2012, IEEE Transactions on Circuits and Systems for Video Technology.

[29]  Tardi Tjahjadi,et al.  Robust view-invariant multiscale gait recognition , 2015, Pattern Recognit..

[30]  Qiang Wu,et al.  Coupled Bilinear Discriminant Projection for Cross-View Gait Recognition , 2020, IEEE Transactions on Circuits and Systems for Video Technology.

[31]  Xiaogang Wang,et al.  A Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Qiang Wu,et al.  Multiple views gait recognition using View Transformation Model based on optimized Gait Energy Image , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[33]  Hongming Shan,et al.  Multi-Task GANs for View-Specific Feature Learning in Gait Recognition , 2019, IEEE Transactions on Information Forensics and Security.

[34]  Haiping Lu,et al.  Uncorrelated Multilinear Discriminant Analysis With Regularization and Aggregation for Tensor Object Recognition , 2009, IEEE Transactions on Neural Networks.

[35]  Yasushi Makihara,et al.  Multi-view discriminant analysis with tensor representation and its application to cross-view gait recognition , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[36]  Yasushi Makihara,et al.  Cross-view gait recognition using view-dependent discriminative analysis , 2014, IEEE International Joint Conference on Biometrics.

[37]  Jiwen Lu,et al.  Uncorrelated discriminant simplex analysis for view-invariant gait signal computing , 2010, Pattern Recognit. Lett..

[38]  Yasushi Makihara,et al.  Gait Recognition Using a View Transformation Model in the Frequency Domain , 2006, ECCV.

[39]  Gang Wang,et al.  Human Identity and Gender Recognition From Gait Sequences With Arbitrary Walking Directions , 2014, IEEE Transactions on Information Forensics and Security.

[40]  Yasushi Makihara,et al.  On Input/Output Architectures for Convolutional Neural Network-Based Cross-View Gait Recognition , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[41]  Qiang Wu,et al.  Coupled Patch Alignment for Matching Cross-View Gaits , 2019, IEEE Transactions on Image Processing.

[42]  Haiping Lu,et al.  A survey of multilinear subspace learning for tensor data , 2011, Pattern Recognit..

[43]  Yasushi Makihara,et al.  Gait-Based Person Recognition Using Arbitrary View Transformation Model , 2015, IEEE Transactions on Image Processing.

[44]  Fan Guo,et al.  Robust Arbitrary-View Gait Recognition Based on 3D Partial Similarity Matching , 2017, IEEE Transactions on Image Processing.

[45]  Haiping Lu,et al.  MPCA: Multilinear Principal Component Analysis of Tensor Objects , 2008, IEEE Transactions on Neural Networks.

[46]  Rui Yan,et al.  An improved biometrics technique based on metric learning approach , 2012, Neurocomputing.

[47]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Andrew Beng Jin Teoh,et al.  A Grassmannian Approach to Address View Change Problem in Gait Recognition , 2017, IEEE Transactions on Cybernetics.

[49]  Haiping Lu,et al.  Uncorrelated Multilinear Principal Component Analysis for Unsupervised Multilinear Subspace Learning , 2009, IEEE Transactions on Neural Networks.

[50]  Hua Li,et al.  3D gait recognition using multiple cameras , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[51]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[52]  Jane You,et al.  Sparse tensor discriminative locality alignment for gait recognition , 2016, 2016 International Joint Conference on Neural Networks (IJCNN).

[53]  Peng Zhang,et al.  Gait recognition and micro-expression recognition based on maximum margin projection with tensor representation , 2016, Neural Computing and Applications.

[54]  Dong Xu,et al.  Discriminant analysis with tensor representation , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[55]  Yasushi Makihara,et al.  View Transformation Model Incorporating Quality Measures for Cross-View Gait Recognition , 2016, IEEE Transactions on Cybernetics.

[56]  Qiang Wu,et al.  Recognizing Gaits Across Views Through Correlated Motion Co-Clustering , 2014, IEEE Transactions on Image Processing.