VersatileGait: A Large-Scale Synthetic Gait Dataset Towards in-the-Wild Simulation

Gait recognition has a rapid development in recent years. However, gait recognition in the wild is not well explored yet. An obvious reason could be ascribed to the lack of diverse training data from the perspective of intrinsic and extrinsic factors. To remedy this problem, we propose to construct a largescale gait dataset with the help of controllable computer simulation. In detail, to diversify the intrinsic factors of gait, we generate numerous characters with diverse attributes and empower them with various types of walking styles. To diversify the extrinsic factors of gait, we build a complicated scene with a dense camera layout. Finally, we design an automated generation toolkit under Unity3D for simulating the walking scenario and capturing the gait data automatically. As a result, we obtain an in-the-wild gait dataset, called VersatileGait, which has more than one million silhouette sequences of 10,000 subjects with diverse scenarios. VersatileGait possesses several nice properties, including huge dataset size, diverse pedestrian attributes, complicated camera layout, high-quality annotations, small domain gap with the real one, good scalability for new demands, and no privacy issues. Based on VersatileGait, we propose series of experiments and applications for both research exploration of gait in the wild and practical applications. Our dataset and its corresponding generation toolkit will be publicly available for further studies.

[1]  Slawomir Bak,et al.  Domain Adaptation through Synthesis for Unsupervised Person Re-identification , 2018, ECCV.

[2]  Shamik Sural,et al.  Frontal Gait Recognition From Incomplete Sequences Using RGB-D Camera , 2014, IEEE Transactions on Information Forensics and Security.

[3]  Yasushi Yagi,et al.  Gait Recognition via Semi-supervised Disentangled Representation Learning to Identity and Covariate Features , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Wei Jia,et al.  Survey of Gait Recognition , 2009, ICIC.

[5]  Cordelia Schmid,et al.  Learning from Synthetic Humans , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Shaogang Gong,et al.  Deep Learning Logo Detection with Data Expansion by Synthesising Context , 2016, 2017 IEEE Winter Conference on Applications of Computer Vision (WACV).

[7]  Ling Shao,et al.  Surpassing Real-World Source Training Data: Random 3D Characters for Generalizable Person Re-Identification , 2020, ACM Multimedia.

[8]  Ewout W Steyerberg,et al.  Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints , 2014, BMC Medical Research Methodology.

[9]  Yi Yang,et al.  Taking a Closer Look at Domain Shift: Category-Level Adversaries for Semantics Consistent Domain Adaptation , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Huiyu Zhou,et al.  Multimodal Gait Recognition for Neurodegenerative Diseases , 2021, IEEE Transactions on Cybernetics.

[11]  Varun Jampani,et al.  Training Deep Networks with Synthetic Data: Bridging the Reality Gap by Domain Randomization , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[12]  Mark S. Nixon,et al.  Self-Calibrating View-Invariant Gait Biometrics , 2010, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  Tieniu Tan,et al.  Efficient Night Gait Recognition Based on Template Matching , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[14]  Shaogang Gong,et al.  Gait recognition using Gait Entropy Image , 2009, ICDP.

[15]  Shiqi Yu,et al.  A model-based gait recognition method with body pose and human prior knowledge , 2020, Pattern Recognit..

[16]  Shunli Zhang,et al.  Gait Recognition with Multiple-Temporal-Scale 3D Convolutional Neural Network , 2020, ACM Multimedia.

[17]  Hua Li,et al.  3D gait recognition using multiple cameras , 2006, 7th International Conference on Automatic Face and Gesture Recognition (FGR06).

[18]  Kurt Keutzer,et al.  Multi-source Domain Adaptation for Semantic Segmentation , 2019, NeurIPS.

[19]  Arun Ross,et al.  Biometric recognition by gait: A survey of modalities and features , 2018, Comput. Vis. Image Underst..

[20]  Yasushi Makihara,et al.  Multi-view large population gait dataset and its performance evaluation for cross-view gait recognition , 2018, IPSJ Transactions on Computer Vision and Applications.

[21]  Yasushi Makihara,et al.  The OU-ISIR Gait Database Comprising the Large Population Dataset and Performance Evaluation of Gait Recognition , 2012, IEEE Transactions on Information Forensics and Security.

[22]  Alireza Sepas-Moghaddam,et al.  Deep Gait Recognition: A Survey , 2021, ArXiv.

[23]  Philip David,et al.  Domain Adaptation for Semantic Segmentation of Urban Scenes , 2017 .

[24]  MaYi,et al.  A New View-Invariant Feature for Cross-View Gait Recognition , 2013 .

[25]  Yongzhen Huang,et al.  Set Residual Network for Silhouette-Based Gait Recognition , 2021, IEEE Transactions on Biometrics, Behavior, and Identity Science.

[26]  Somaya Al-Máadeed,et al.  Robust gait recognition: a comprehensive survey , 2018, IET Biom..

[27]  Tieniu Tan,et al.  A Framework for Evaluating the Effect of View Angle, Clothing and Carrying Condition on Gait Recognition , 2006, 18th International Conference on Pattern Recognition (ICPR'06).

[28]  Alexander G. Schwing,et al.  SAIL-VOS: Semantic Amodal Instance Level Video Object Segmentation – A Synthetic Dataset and Baselines , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  Yasushi Makihara,et al.  Gait Recognition Using a View Transformation Model in the Frequency Domain , 2006, ECCV.

[30]  Xiang Li,et al.  The OU-ISIR Gait Database comprising the Large Population Dataset with Age and performance evaluation of age estimation , 2017, IPSJ Transactions on Computer Vision and Applications.

[31]  Shamik Sural,et al.  Gait Recognition in the Presence of Occlusion: A New Dataset and Baseline Algorithms , 2011 .

[32]  Qing Li,et al.  GaitPart: Temporal Part-Based Model for Gait Recognition , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Björn W. Schuller,et al.  The TUM Gait from Audio, Image and Depth (GAID) database: Multimodal recognition of subjects and traits , 2014, J. Vis. Commun. Image Represent..

[34]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Simone Re,et al.  Ideas and methods for modeling 3D human figures: the principal algorithms used by MakeHuman and their implementation in a new approach to parametric modeling , 2008, Bangalore Compute Conf..

[36]  Marc T. Law,et al.  Sim2SG: Sim-to-Real Scene Graph Generation for Transfer Learning , 2020, ArXiv.

[37]  Nikolaos V. Boulgouris,et al.  Gait Recognition Based on Human Body Components , 2007, 2007 IEEE International Conference on Image Processing.

[38]  Tieniu Tan,et al.  Gait recognition based on Procrustes shape analysis , 2002, Proceedings. International Conference on Image Processing.

[39]  Arun Ross,et al.  Investigating gait recognition in the short-wave infrared (SWIR) spectrum: dataset and challenges , 2013, Defense, Security, and Sensing.

[40]  Osama Masoud,et al.  View-independent human motion classification using image-based reconstruction , 2009, Image Vis. Comput..

[41]  Sebastian Ruder,et al.  An Overview of Multi-Task Learning in Deep Neural Networks , 2017, ArXiv.

[42]  Imed Bouchrika,et al.  A Survey of Using Biometrics for Smart Visual Surveillance: Gait Recognition , 2018 .

[43]  Andrea Palazzi,et al.  Learning to Detect and Track Visible and Occluded Body Joints in a Virtual World , 2018, ECCV.

[44]  Wei Lin,et al.  Learning From Synthetic Data for Crowd Counting in the Wild , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Davrondzhon Gafurov,et al.  A Survey of Biometric Gait Recognition: Approaches, Security and Challenges , 2007 .

[46]  Zihan Zhou,et al.  Structured3D: A Large Photo-realistic Dataset for Structured 3D Modeling , 2019, ECCV.

[47]  Yasushi Makihara,et al.  End-to-End Model-Based Gait Recognition , 2020, ACCV.

[48]  Yasushi Makihara,et al.  Cross-View Gait Recognition Using Pairwise Spatial Transformer Networks , 2021, IEEE Transactions on Circuits and Systems for Video Technology.

[49]  Jasvinder Pal Singh,et al.  A Multi-Gait Dataset for Human Recognition under Occlusion Scenario , 2019, 2019 International Conference on Issues and Challenges in Intelligent Computing Techniques (ICICT).

[50]  Qi Tian,et al.  Scalable Person Re-identification: A Benchmark , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[51]  Swami Sankaranarayanan,et al.  Learning from Synthetic Data: Addressing Domain Shift for Semantic Segmentation , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[52]  K. Sugandhi,et al.  Feature Extraction Methods for Human Gait Recognition – A Survey , 2016 .

[53]  Yongzhen Huang,et al.  Gait Lateral Network: Learning Discriminative and Compact Representations for Gait Recognition , 2020, ECCV.

[54]  Sudeep Sarkar,et al.  The humanID gait challenge problem: data sets, performance, and analysis , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Jianfeng Feng,et al.  GaitSet: Regarding Gait as a Set for Cross-View Gait Recognition , 2018, AAAI.

[56]  Xiaoming Liu,et al.  Gait Recognition via Disentangled Representation Learning , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[57]  D.Sasikala M.Pushparani,et al.  A Survey of Gait Recognition Approaches Using PCA and ICA , 2012 .

[58]  Gunawan Ariyanto,et al.  Model-based 3D gait biometrics , 2011, 2011 International Joint Conference on Biometrics (IJCB).

[59]  Mark S. Nixon,et al.  Performing content-based retrieval of humans using gait biometrics , 2008, Multimedia Tools and Applications.

[60]  Qiang Wu,et al.  Recognizing Gaits Across Views Through Correlated Motion Co-Clustering , 2014, IEEE Transactions on Image Processing.

[61]  Ralph Gross,et al.  The CMU Motion of Body (MoBo) Database , 2001 .

[62]  Yuzhuo Fu,et al.  Unsupervised Domain Adaptation Through Synthesis For Person Re-Identification , 2020, 2020 IEEE International Conference on Multimedia and Expo (ICME).

[63]  Xiang Li,et al.  Joint Intensity and Spatial Metric Learning for Robust Gait Recognition , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[64]  Kyoobin Lee,et al.  Feature Extraction Using an RNN Autoencoder for Skeleton-Based Abnormal Gait Recognition , 2020, IEEE Access.