Automatic Health Problem Detection from Gait Videos Using Deep Neural Networks

The aim of this study is developing an automatic system for detection of gait-related health problems using Deep Neural Networks (DNNs). The proposed system takes a video of patients as the input and estimates their 3D body pose using a DNN based method. Our code is publicly available at this https URL. The resulting 3D body pose time series are then analyzed in a classifier, which classifies input gait videos into four different groups including Healthy, with Parkinsons disease, Post Stroke patient, and with orthopedic problems. The proposed system removes the requirement of complex and heavy equipment and large laboratory space, and makes the system practical for home use. Moreover, it does not need domain knowledge for feature engineering since it is capable of extracting semantic and high level features from the input data. The experimental results showed the classification accuracy of 56% to 96% for different groups. Furthermore, only 1 out of 25 healthy subjects were misclassified (False positive), and only 1 out of 70 patients were classified as a healthy subject (False negative). This study presents a starting point toward a powerful tool for automatic classification of gait disorders and can be used as a basis for future applications of Deep Learning in clinical gait analysis. Since the system uses digital cameras as the only required equipment, it can be employed in domestic environment of patients and elderly people for consistent gait monitoring and early detection of gait alterations.

[1]  Cristian Sminchisescu,et al.  Twin Gaussian Processes for Structured Prediction , 2010, International Journal of Computer Vision.

[2]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[3]  Matjaz Gams,et al.  Automatic recognition of gait-related health problems in the elderly using machine learning , 2012, Multimedia Tools and Applications.

[4]  Varun Ramakrishna,et al.  Convolutional Pose Machines , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Nooritawati Md Tahir,et al.  Anomaly gait classification of Parkinson disease based on ANN , 2011, 2011 IEEE International Conference on System Engineering and Technology.

[6]  Christian Wolf,et al.  Sequential Deep Learning for Human Action Recognition , 2011, HBU.

[7]  Nojun Kwak,et al.  3D Human Pose Estimation Using Convolutional Neural Networks with 2D Pose Information , 2016, ECCV Workshops.

[8]  Xiaoli Li,et al.  Deep Convolutional Neural Networks on Multichannel Time Series for Human Activity Recognition , 2015, IJCAI.

[9]  James J. Little,et al.  A Simple Yet Effective Baseline for 3d Human Pose Estimation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[10]  Tim Oates,et al.  Time series classification from scratch with deep neural networks: A strong baseline , 2016, 2017 International Joint Conference on Neural Networks (IJCNN).

[11]  Miad Faezipour,et al.  Desynchronization and Energy Efficiency of Gaussian Neurostimulation on Different Sites of the Basal Ganglia , 2017, 2017 IEEE 17th International Conference on Bioinformatics and Bioengineering (BIBE).

[12]  Bernt Schiele,et al.  Multi-view Pictorial Structures for 3D Human Pose Estimation , 2013, BMVC.

[13]  Jia Deng,et al.  Stacked Hourglass Networks for Human Pose Estimation , 2016, ECCV.

[14]  Angelo M. Sabatini,et al.  A Machine Learning Framework for Gait Classification Using Inertial Sensors: Application to Elderly, Post-Stroke and Huntington’s Disease Patients , 2016, Sensors.

[15]  Claudio Cobelli,et al.  Markerless analysis of front crawl swimming. , 2011, Journal of biomechanics.

[16]  Xu Xu,et al.  Predicting 3-D Lower Back Joint Load in Lifting: A Deep Pose Estimation Approach , 2019, IEEE Transactions on Human-Machine Systems.

[17]  Miad Faezipour,et al.  Towards frequency adaptation for delayed feedback deep brain stimulations , 2018, Neural regeneration research.

[18]  Cristian Sminchisescu,et al.  Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[19]  Xu Xu,et al.  A Deep Neural Network-based method for estimation of 3D lifting motions. , 2019, Journal of biomechanics.

[20]  M Illert,et al.  Comparative analysis of the gait disorder of normal pressure hydrocephalus and Parkinson's disease , 2001, Journal of neurology, neurosurgery, and psychiatry.

[21]  Xu Xu,et al.  Toward Marker-Free 3D Pose Estimation in Lifting: A Deep Multi-View Solution , 2018, 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018).

[22]  Robert LeMoyne,et al.  Implementation of Machine Learning for Classifying Hemiplegic Gait Disparity through Use of a Force Plate , 2014, 2014 13th International Conference on Machine Learning and Applications.

[23]  Nasser M. Nasrabadi,et al.  Deep Cross Polarimetric Thermal-to-Visible Face Recognition , 2018, 2018 International Conference on Biometrics (ICB).

[24]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[25]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[26]  Xiaowei Zhou,et al.  Sparseness Meets Deepness: 3D Human Pose Estimation from Monocular Video , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  Xiaowei Zhou,et al.  Coarse-to-Fine Volumetric Prediction for Single-Image 3D Human Pose , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[28]  C. Cobelli,et al.  A Markerless Motion Capture System to Study Musculoskeletal Biomechanics: Visual Hull and Simulated Annealing Approach , 2006, Annals of Biomedical Engineering.

[29]  Nasser M. Nasrabadi,et al.  Deep Sketch-Photo Face Recognition Assisted by Facial Attributes , 2018, 2018 IEEE 9th International Conference on Biometrics Theory, Applications and Systems (BTAS).

[30]  Wenyao Xu,et al.  A pervasive and sensor-free Deep Learning system for Parkinsonian gait analysis , 2018, 2018 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI).

[31]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Xiaowei Zhou,et al.  Harvesting Multiple Views for Marker-Less 3D Human Pose Annotations , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[33]  Miad Faezipour,et al.  Computational Stimulation of the Basal Ganglia Neurons with Cost Effective Delayed Gaussian Waveforms , 2017, Front. Comput. Neurosci..

[34]  Begonya Garcia-Zapirain,et al.  Gait Analysis Methods: An Overview of Wearable and Non-Wearable Systems, Highlighting Clinical Applications , 2014, Sensors.

[35]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[36]  Pascal Fua,et al.  Learning to Fuse 2D and 3D Image Cues for Monocular Body Pose Estimation , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[37]  Y. S. Rao,et al.  SVM based machine learning approach to identify Parkinson's disease using gait analysis , 2016, 2016 International Conference on Inventive Computation Technologies (ICICT).