Applying adversarial auto-encoder for estimating human walking gait abnormality index

This paper proposes an approach that estimates a human walking gait abnormality index using an adversarial auto-encoder (AAE), i.e., a combination of auto-encoder and generative adversarial network (GAN). Since most GAN-based models have been employed as data generators, our work introduces another perspective of their application. This method directly works on a sequence of 3D point clouds representing the walking postures of a subject. By fitting a cylinder onto each point cloud and feeding cylindrical histograms to an appropriate AAE, our system is able to provide different measures that may be used as gait abnormality indices. The combinations of such quantities are also investigated to obtain improved indicators. The ability of our method is demonstrated by experimenting on a large dataset of nearly 100 thousands point clouds, and the results outperform related approaches that employ different input data types.

[1]  John Cosmas,et al.  Recognition of postures and Freezing of Gait in Parkinson's disease patients using Microsoft Kinect sensor , 2015, 2015 7th International IEEE/EMBS Conference on Neural Engineering (NER).

[2]  Jean Meunier,et al.  3D Reconstruction With Time-of-Flight Depth Camera and Multiple Mirrors , 2018, IEEE Access.

[3]  James W. Davis.  Hierarchical motion history images for recognizing human motion , 2001, Proceedings IEEE Workshop on Detection and Recognition of Events in Video.

[4]  Navdeep Jaitly,et al.  Adversarial Autoencoders , 2015, ArXiv.

[5]  Eui Chul Lee,et al.  Pointing Gesture Interface for Large Display Environments Based on the Kinect Skeleton Model , 2014 .

[6]  Jean Meunier,et al.  Using ToF camera and two mirrors for 3D reconstruction of dynamic objects , 2018 .

[7]  Enrico Tronci,et al.  Electric Power System Anomaly Detection Using Neural Networks , 2004, KES.

[8]  Lu Qin,et al.  Movement Disorder Detection via Adaptively Fused Gait Analysis Based on Kinect Sensors , 2018, IEEE Sensors Journal.

[9]  Andrew W. Fitzgibbon,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR 2011.

[10]  Keith M. Kendrick,et al.  Gait Rhythm Fluctuation Analysis for Neurodegenerative Diseases by Empirical Mode Decomposition , 2017, IEEE Transactions on Biomedical Engineering.

[11]  Jean Meunier,et al.  Extracting silhouette-based characteristics for human gait analysis using one camera , 2014, SoICT.

[12]  Franck Multon,et al.  New Lower-Limb Gait Asymmetry Indices Based on a Depth Camera , 2015, Sensors.

[13]  Alexandros André Chaaraoui,et al.  Abnormal gait detection with RGB-D devices using joint motion history features , 2015, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).

[14]  A. K. Karunakar,et al.  Classification of gait signals into different neurodegenerative diseases using statistical analysis and recurrence quantification analysis , 2020, Pattern Recognit. Lett..

[15]  Franck Multon,et al.  Multiple depth cameras calibration and body volume reconstruction for gait analysis , 2012, 2012 11th International Conference on Information Science, Signal Processing and their Applications (ISSPA).

[16]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[17]  Jean Meunier,et al.  Walking gait dataset : point clouds , skeletons and silhouettes Technical Report Number 1379 , 2018 .

[18]  Wei Liu,et al.  Latent Max-Margin Multitask Learning With Skelets for 3-D Action Recognition , 2017, IEEE Transactions on Cybernetics.

[19]  Jorge López,et al.  Identification of Multimodal Human-Robot Interaction Using Combined Kernels , 2015, IBICA.

[20]  Bir Bhanu,et al.  Individual recognition using gait energy image , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Nathan Srebro,et al.  The Marginal Value of Adaptive Gradient Methods in Machine Learning , 2017, NIPS.

[22]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[23]  N. Huang,et al.  The empirical mode decomposition and the Hilbert spectrum for nonlinear and non-stationary time series analysis , 1998, Proceedings of the Royal Society of London. Series A: Mathematical, Physical and Engineering Sciences.

[24]  Jean Meunier,et al.  Skeleton-Based Abnormal Gait Detection , 2016, Sensors.

[25]  Sebastian Nowozin,et al.  Stabilizing Training of Generative Adversarial Networks through Regularization , 2017, NIPS.

[26]  John K. Tsotsos,et al.  Automatic detection of abnormal gait , 2009, Image Vis. Comput..

[27]  N. Marwan,et al.  Recurrence quantification analysis : theory and best practices , 2015 .

[28]  Yuanyuan Zhang,et al.  Real Time Gait Recognition System Based on Kinect Skeleton Feature , 2014, ACCV Workshops.

[29]  Takehisa Yairi,et al.  Anomaly Detection Using Autoencoders with Nonlinear Dimensionality Reduction , 2014, MLSDA'14.

[30]  Jean Meunier,et al.  Matching-based depth camera and mirrors for 3D reconstruction , 2018, Commercial + Scientific Sensing and Imaging.

[31]  Xinbo Gao,et al.  Multi-task human action recognition via exploring super-category , 2016, Signal Process..

[32]  David López-Fernández,et al.  A new approach for multi-view gait recognition on unconstrained paths , 2016, J. Vis. Commun. Image Represent..

[33]  Wei Liu,et al.  Discriminative Multi-instance Multitask Learning for 3D Action Recognition , 2017, IEEE Transactions on Multimedia.

[34]  Tae-Kyun Kim,et al.  Real-time Action Recognition by Spatiotemporal Semantic and Structural Forests , 2010, BMVC.

[35]  Jean Meunier,et al.  Assessment of gait normality using a depth camera and mirrors , 2018, 2018 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI).