Deep Metric Learning for Scalable Gait-Based Person Re-Identification Using Force Platform Data

Walking gait data acquired with force platforms may be used for person re-identification (re-ID) in various authentication, surveillance, and forensics applications. Current force platform-based re-ID systems classify a fixed set of identities (IDs), which presents a problem when IDs are added or removed from the database. We formulated force platform-based re-ID as a deep metric learning (DML) task, whereby a deep neural network learns a feature representation that can be compared between inputs using a distance metric. The force platform dataset used in this study is one of the largest and the most comprehensive of its kind, containing 193 IDs with significant variations in clothing, footwear, walking speed, and time between trials. Several DML model architectures were evaluated in a challenging setting where none of the IDs were seen during training (i.e., zero-shot re-ID) and there was only one prior sample per ID to compare with each query sample. The best architecture was 85% accurate in this setting, though an analysis of changes in walking speed and footwear between measurement instances revealed that accuracy was 28% higher on same-speed, same-footwear comparisons, compared to cross-speed, cross-footwear comparisons. These results demonstrate the potential of DML algorithms for zero-shot re-ID using force platform data, and highlight challenging cases.

[1]  M. Derlatka,et al.  Real-world measurements of ground reaction forces of normal gait of young adults wearing various footwear , 2023, Scientific Data.

[2]  M. Borowska,et al.  Ensemble of Heterogeneous Base Classifiers for Human Gait Recognition , 2023, Sensors.

[3]  Yongzhen Huang,et al.  CASIA-E: A Large Comprehensive Dataset for Gait Recognition , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Jie Zhou,et al.  Gait Recognition in the Wild: A Benchmark , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[5]  W. Schöllhorn,et al.  Gutenberg Gait Database, a ground reaction force database of level overground walking in healthy individuals , 2021, Scientific data.

[6]  Jian Pei,et al.  Model complexity of deep learning: a survey , 2021, Knowledge and Information Systems.

[7]  A. Etemad,et al.  Deep Gait Recognition: A Survey , 2021, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  G. Rigoll,et al.  Gaitgraph: Graph Convolutional Network for Skeleton-Based Gait Recognition , 2021, 2021 IEEE International Conference on Image Processing (ICIP).

[9]  Yasushi Makihara,et al.  Cross-View Gait Recognition Using Pairwise Spatial Transformer Networks , 2021, IEEE Transactions on Circuits and Systems for Video Technology.

[10]  Ali Etemad,et al.  View-Invariant Gait Recognition With Attentive Recurrent Learning of Partial Representations , 2020, IEEE Transactions on Biometrics, Behavior, and Identity Science.

[11]  Nikolaus F. Troje,et al.  Gait Recognition using Multi-Scale Partial Representation Transformation with Capsules , 2020, 2020 25th International Conference on Pattern Recognition (ICPR).

[12]  Marcin Derlatka,et al.  Time Removed Repeated Trials to Test the Quality of a Human Gait Recognition System , 2020, CISIM.

[13]  Shunli Zhang,et al.  Gait Recognition with Multiple-Temporal-Scale 3D Convolutional Neural Network , 2020, ACM Multimedia.

[14]  Jan Andrysek,et al.  Two-dimensional video gait analysis: A systematic review of reliability, validity, and best practice considerations , 2020, Prosthetics and orthotics international.

[15]  Qing Li,et al.  GaitPart: Temporal Part-Based Model for Gait Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Brian Horsak,et al.  GaitRec, a large-scale ground reaction force dataset of healthy and impaired gait , 2020, Scientific Data.

[17]  Shiqi Yu,et al.  A model-based gait recognition method with body pose and human prior knowledge , 2020, Pattern Recognit..

[18]  W. Schöllhorn,et al.  Systematic Comparison of the Influence of Different Data Preprocessing Methods on the Performance of Gait Classifications Using Machine Learning , 2019, Frontiers in Bioengineering and Biotechnology.

[19]  Dominic Thewlis,et al.  Simulating Time-Series Data for Improved Deep Neural Network Performance , 2019, IEEE Access.

[20]  Hasan Şakir Bilge,et al.  Deep Metric Learning: A Survey , 2019, Symmetry.

[21]  Philippe Terrier,et al.  Gait recognition via deep learning of the center-of-pressure trajectory , 2019, Applied Sciences.

[22]  Yasushi Makihara,et al.  Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition , 2018, IPSJ Transactions on Computer Vision and Applications.

[23]  Jianfeng Feng,et al.  GaitSet: Regarding Gait as a Set for Cross-View Gait Recognition , 2018, AAAI.

[24]  Vir V. Phoha,et al.  A Survey on Gait Recognition , 2018, ACM Comput. Surv..

[25]  Klaus-Robert Müller,et al.  Explaining the unique nature of individual gait patterns with deep learning , 2018, Scientific Reports.

[26]  Alireza Rastegarpanah,et al.  Targeting effect on gait parameters in healthy individuals and post-stroke hemiparetic individuals , 2018, Journal of rehabilitation and assistive technologies engineering.

[27]  Sashank J. Reddi,et al.  On the Convergence of Adam and Beyond , 2018, ICLR.

[28]  Arun Ross,et al.  Biometric recognition by gait: A survey of modalities and features , 2018, Comput. Vis. Image Underst..

[29]  Derlatka Marcin,et al.  Human gait recognition based on ground reaction forces in case of sport shoes and high heels , 2017, 2017 IEEE International Conference on INnovations in Intelligent SysTems and Applications (INISTA).

[30]  Lucas Beyer,et al.  In Defense of the Triplet Loss for Person Re-Identification , 2017, ArXiv.

[31]  Xiaogang Wang,et al.  A Comprehensive Study on Cross-View Gait Based Human Identification with Deep CNNs , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[32]  Kevin Gimpel,et al.  Gaussian Error Linear Units (GELUs) , 2016, 1606.08415.

[33]  Yasushi Makihara,et al.  GEINet: View-invariant gait recognition using a convolutional neural network , 2016, 2016 International Conference on Biometrics (ICB).

[34]  Zheng Wang,et al.  Zero-Shot Person Re-identification via Cross-View Consistency , 2016, IEEE Transactions on Multimedia.

[35]  Sepp Hochreiter,et al.  Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs) , 2015, ICLR.

[36]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[37]  Bastiaan R. Bloem,et al.  Prevalence and Burden of Gait Disorders in Elderly Men and Women Aged 60–97 Years: A Population-Based Study , 2013, PloS one.

[38]  Yasushi Makihara,et al.  The OU-ISIR Gait Database Comprising the Large Population Dataset and Performance Evaluation of Gait Recognition , 2012, IEEE Transactions on Information Forensics and Security.

[39]  J. Theocharis,et al.  Feature selection based on a fuzzy complementary criterion: application to gait recognition using ground reaction forces , 2012, Computer methods in biomechanics and biomedical engineering.

[40]  Marcin Derlatka,et al.  Human Gait Recognition Based on Signals from Two Force Plates , 2012, ICAISC.

[41]  Mark S. Nixon,et al.  The Effect of Time on Gait Recognition Performance , 2012, IEEE Transactions on Information Forensics and Security.

[42]  Ming Ye,et al.  Effect of unilateral load carriage on postures and gait symmetry in ground reaction force during walking , 2010, Computer methods in biomechanics and biomedical engineering.

[43]  J Maxwell Donelan,et al.  Dynamic Principles of Gait and Their Clinical Implications , 2010, Physical Therapy.

[44]  Arun Ross,et al.  Handbook of Biometrics , 2007 .

[45]  Carla Schlatter Ellis,et al.  Using Ground Reaction Forces from Gait Analysis: Body Mass as a Weak Biometric , 2007, Pervasive.

[46]  Tieniu Tan,et al.  A Framework for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[47]  Lei Ren,et al.  Dynamic analysis of load carriage biomechanics during level walking. , 2005, Journal of biomechanics.

[48]  Nachiappan Chockalingam,et al.  Do strain gauge force platforms need in situ correction? , 2002, Gait & posture.

[49]  Philippe C. Cattin,et al.  Sensor fusion for a biometric system using gait , 2001, Conference Documentation International Conference on Multisensor Fusion and Integration for Intelligent Systems. MFI 2001 (Cat. No.01TH8590).

[50]  S. Stevenage,et al.  Visual analysis of gait as a cue to identity , 1999 .

[51]  K. An,et al.  Estimate of the Optimum Cutoff Frequency for the Butterworth Low-Pass Digital Filter , 1999 .

[52]  S. Simon.  Gait Analysis, Normal and Pathological Function. , 1993 .

[53]  Nancy Chinchor,et al.  MUC-4 evaluation metrics , 1992, MUC.

[54]  D. Winter,et al.  Assessment of balance control in humans. , 1990, Medical progress through technology.

[55]  Yasushi Makihara,et al.  End-to-End Model-Based Gait Recognition , 2020, ACCV.

[56]  Yongzhen Huang,et al.  Gait Lateral Network: Learning Discriminative and Compact Representations for Gait Recognition , 2020, ECCV.

[57]  Mario Baum,et al.  Handbook Of Biometrics , 2016 .

[58]  Katsushi Ikeuchi,et al.  Automatic Gait Recognition , 2014, Computer Vision, A Reference Guide.

[59]  Danqi Chen,et al.  of the Association for Computational Linguistics: , 2001 .

[60]  J B King,et al.  Gait Analysis. An Introduction , 1992 .

[61]  M. P. Murray.  Gait as a total pattern of movement. , 1967, American journal of physical medicine.