GaitFormer: Revisiting Intrinsic Periodicity for Gait Recognition

Gait recognition aims to distinguish different walking patterns by analyzing video-level human silhouettes, rather than relying on appearance information. Previous research on gait recognition has primarily focused on extracting local or global spatial-temporal representations, while overlooking the intrinsic periodic features of gait sequences, which, when fully utilized, can significantly enhance performance. In this work, we propose a plug-and-play strategy, called Temporal Periodic Alignment (TPA), which leverages the periodic nature and fine-grained temporal dependencies of gait patterns. The TPA strategy comprises two key components. The first component is Adaptive Fourier-transform Position Encoding (AFPE), which adaptively converts features and discrete-time signals into embeddings that are sensitive to periodic walking patterns. The second component is the Temporal Aggregation Module (TAM), which separates embeddings into trend and seasonal components, and extracts meaningful temporal correlations to identify primary components, while filtering out random noise. We present a simple and effective baseline method for gait recognition, based on the TPA strategy. Extensive experiments conducted on three popular public datasets (CASIA-B, OU-MVLP, and GREW) demonstrate that our proposed method achieves state-of-the-art performance on multiple benchmark tests.

[1]  Wei Su,et al.  MetaGait: Learning to Learn an Omni Sample Adaptive Representation for Gait Recognition , 2023, ECCV.

[2]  Yongzhen Huang,et al.  OpenGait: Revisiting Gait Recognition Toward Better Practicality , 2022, 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Liang Wang,et al.  A Comprehensive Survey on Deep Gait Recognition: Algorithms, Datasets and Challenges , 2022, ArXiv.

[4]  Annan Li,et al.  Lagrange Motion Analysis and View Embeddings for Improved Gait Recognition , 2022, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Shangce Gao,et al.  TransGait: Multimodal-based gait recognition with set transformer , 2022, Applied Intelligence.

[6]  Yasushi Makihara,et al.  Multi-View Large Population Gait Database With Human Meshes and Its Performance Evaluation , 2022, IEEE Transactions on Biometrics, Behavior, and Identity Science.

[7]  Jian Sun,et al.  PETR: Position Embedding Transformation for Multi-View 3D Object Detection , 2022, ECCV.

[8]  Yongzhen Huang,et al.  GaitEdge: Beyond Plain End-to-end Gait Recognition for Better Practicality , 2022, ECCV.

[9]  Shiqi Yu,et al.  Gait Recognition with Mask-based Regularization , 2022, ArXiv.

[10]  Jie Zhou,et al.  Gait Recognition in the Wild: A Benchmark , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[11]  Xinmei Tian,et al.  3D Local Convolutional Neural Networks for Gait Recognition , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Xinggang Wang,et al.  Context-Sensitive Temporal Feature Learning for Gait Recognition , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[13]  Stephen Lin,et al.  Video Swin Transformer , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Chunhua Shen,et al.  Conditional Positional Encodings for Vision Transformers , 2021, ICLR.

[15]  A. Etemad,et al.  Deep Gait Recognition: A Survey , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Xin Yu,et al.  Gait Recognition via Effective Global-Local Feature Representation and Local Temporal Aggregation , 2020, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[17]  S. Gelly,et al.  An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale , 2020, ICLR.

[18]  Yang Yu,et al.  Performance Evaluation of Model-Based Gait on Multi-View Very Large Population Database With Pose Sequences , 2020, IEEE Transactions on Biometrics, Behavior, and Identity Science.

[19]  Qing Li,et al.  GaitPart: Temporal Part-Based Model for Gait Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[20]  Shiqi Yu,et al.  A model-based gait recognition method with body pose and human prior knowledge , 2020, Pattern Recognit..

[21]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[22]  Yasushi Makihara,et al.  On Input/Output Architectures for Convolutional Neural Network-Based Cross-View Gait Recognition , 2019, IEEE Transactions on Circuits and Systems for Video Technology.

[23]  Yasushi Makihara,et al.  Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition , 2018, IPSJ Transactions on Computer Vision and Applications.

[24]  Jianfeng Feng,et al.  GaitSet: Regarding Gait as a Set for Cross-View Gait Recognition , 2018, AAAI.

[25]  Jean Meunier,et al.  Skeleton-based Gait Index Estimation with LSTMs , 2018, 2018 IEEE/ACIS 17th International Conference on Computer and Information Science (ICIS).

[26]  Arun Ross,et al.  Biometric recognition by gait: A survey of modalities and features , 2018, Comput. Vis. Image Underst..

[27]  Stefanos Zafeiriou,et al.  ArcFace: Additive Angular Margin Loss for Deep Face Recognition , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  Ying Li,et al.  View-invariant gait recognition method by three-dimensional convolutional neural network , 2018 .

[29]  Frank Hutter,et al.  Decoupled Weight Decay Regularization , 2017, ICLR.

[30]  Shiqi Yu,et al.  Pose-Based Temporal-Spatial Network (PTSN) for Gait Recognition with Carrying and Clothing Variations , 2017, CCBR.

[31]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[32]  Limin Wang,et al.  Temporal Segment Networks for Action Recognition in Videos , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[33]  Xiaogang Wang,et al.  A Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Yang Feng,et al.  Learning effective Gait features using LSTM , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[35]  Yasushi Makihara,et al.  GEINet: View-invariant gait recognition using a convolutional neural network , 2016, 2016 International Conference on Biometrics (ICB).

[36]  Wu Liu,et al.  Siamese neural network based gait recognition for human identification , 2016, 2016 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[37]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[38]  Nir Ailon,et al.  Deep Metric Learning Using Triplet Network , 2014, SIMBAD.

[39]  Yasushi Makihara,et al.  Gait Recognition Using Period-Based Phase Synchronization for Low Frame-Rate Videos , 2010, 2010 20th International Conference on Pattern Recognition.

[40]  Tieniu Tan,et al.  Efficient Night Gait Recognition Based on Template Matching , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[41]  Tieniu Tan,et al.  A Framework for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[42]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[43]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[44]  Mark S. Nixon,et al.  Using Gait as a Biometric, via Phase-weighted Magnitude Spectra , 1997, AVBPA.

[45]  Stephen Lin,et al.  Swin Transformer: Hierarchical Vision Transformer using Shifted Windows , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[46]  Yasushi Makihara,et al.  End-to-End Model-Based Gait Recognition , 2020, ACCV.

[47]  Yongzhen Huang,et al.  Gait Lateral Network: Learning Discriminative and Compact Representations for Gait Recognition , 2020, ECCV.