Explaining the unique nature of individual gait patterns with deep learning

Machine learning (ML) techniques such as (deep) artificial neural networks (DNN) are solving very successfully a plethora of tasks and provide new predictive models for complex physical, chemical, biological and social systems. However, in most cases this comes with the disadvantage of acting as a black box, rarely providing information about what made them arrive at a particular prediction. This black box aspect of ML techniques can be problematic especially in medical diagnoses, so far hampering a clinical acceptance. The present paper studies the uniqueness of individual gait patterns in clinical biomechanics using DNNs. By attributing portions of the model predictions back to the input variables (ground reaction forces and full-body joint angles), the Layer-Wise Relevance Propagation (LRP) technique reliably demonstrates which variables at what time windows of the gait cycle are most relevant for the characterisation of gait patterns from a certain individual. By measuring the time-resolved contribution of each input variable to the prediction of ML techniques such as DNNs, our method describes the first general framework that enables to understand and interpret non-linear ML methods in (biomechanical) gait analysis and thereby supplies a powerful tool for analysis, diagnosis and treatment of human gait.

[1]  Alexander Binder,et al.  Analyzing Classifiers: Fisher Vectors and Deep Neural Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Wolfgang I. Schöllhorn,et al.  One-year persistence of individual gait patterns identified in a follow-up study - A call for individualised diagnose and therapy. , 2017, Gait & posture.

[4]  R Begg,et al.  A machine learning approach for automated recognition of movement patterns using basic, kinetic and kinematic gait data. , 2005, Journal of biomechanics.

[5]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[6]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[7]  Reed Ferber,et al.  Biomechanical Deviations During Level Walking Associated With Knee Osteoarthritis: A Systematic Review and Meta‐Analysis , 2013, Arthritis care & research.

[8]  Motoaki Kawanabe,et al.  How to Explain Individual Classification Decisions , 2009, J. Mach. Learn. Res..

[9]  R. Lipton,et al.  Epidemiology of Gait Disorders in Community‐Residing Older Adults , 2006, Journal of the American Geriatrics Society.

[10]  Klaus-Robert Müller,et al.  Interpretable deep neural networks for single-trial EEG classification , 2016, Journal of Neuroscience Methods.

[11]  T Chau,et al.  A review of analytical techniques for gait data. Part 1: Fuzzy, statistical and fractal methods. , 2001, Gait & posture.

[12]  Subashan Perera,et al.  Improvements in Speed-Based Gait Classifications Are Meaningful , 2007, Stroke.

[13]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[14]  Jeffrey Kaye,et al.  The trajectory of gait speed preceding mild cognitive impairment. , 2010, Archives of neurology.

[15]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[16]  Alexander Binder,et al.  Understanding and Comparing Deep Neural Networks for Age and Gender Classification , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[17]  Jeffrey M. Hausdorff,et al.  Classification of gait disturbances: Distinguishing between continuous and episodic changes , 2013, Movement disorders : official journal of the Movement Disorder Society.

[18]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[19]  R. Durbin,et al.  Using probabilistic estimation of expression residuals (PEER) to obtain increased power and interpretability of gene expression analyses , 2012, Nature Protocols.

[20]  Klaus P. Ebmeier,et al.  What can gait tell us about dementia? Review of epidemiological and neuropsychological evidence. , 2017, Gait & posture.

[21]  Meg E. Morris,et al.  Classification of Gait Disorders Following Traumatic Brain Injury , 2015, The Journal of head trauma rehabilitation.

[22]  Cecilia Fagerström,et al.  Mobility, functional ability and health-related quality of life among people of 60 years or older , 2010, Aging clinical and experimental research.

[23]  Benedicte Vanwanseele,et al.  Effect of children's shoes on gait: a systematic review and meta-analysis , 2011, Journal of foot and ankle research.

[24]  Peter Federolf,et al.  A holistic approach to study the temporal variability in gait. , 2012, Journal of biomechanics.

[25]  Alexander Binder,et al.  Evaluating the Visualization of What a Deep Neural Network Has Learned , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[26]  Kathy Martin Measuring Walking: A Handbook of Clinical Gait Analysis , 2014 .

[27]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[28]  M. Gevrey,et al.  Review and comparison of methods to study the contribution of variables in artificial neural network models , 2003 .

[29]  Anthony Widjaja,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2003, IEEE Transactions on Neural Networks.

[30]  Meg E Morris,et al.  Determinants of health-related quality of life in Parkinson's disease: a systematic review. , 2011, Parkinsonism & related disorders.

[31]  T Chau,et al.  A review of analytical techniques for gait data. Part 2: neural network and wavelet methods. , 2001, Gait & posture.

[32]  C. Powers,et al.  Quality Indicators for the Management and Prevention of Falls and Mobility Problems in Vulnerable Elders , 2001, Annals of Internal Medicine.

[33]  Alexander Binder,et al.  The LRP Toolbox for Artificial Neural Networks , 2016, J. Mach. Learn. Res..

[34]  Ralf Mikut,et al.  Automated feature assessment in instrumented gait analysis. , 2006, Gait & posture.

[35]  Carole A. Tucker,et al.  Measuring Walking: A Handbook of Clinical Gait Analysis , 2014 .

[36]  Caterina Pesce,et al.  Health and Quality of Life Perception in Older Adults: The Joint Role of Cognitive Efficiency and Functional Mobility , 2015, International journal of environmental research and public health.

[37]  Benno M. Nigg,et al.  Daily changes of individual gait patterns identified by means of support vector machines. , 2016, Gait & posture.

[38]  Hermann Schwameder,et al.  Computer aided analysis of gait patterns in patients with acute anterior cruciate ligament injury. , 2016, Clinical biomechanics.

[39]  Andrea Vedaldi,et al.  Interpretable Explanations of Black Boxes by Meaningful Perturbation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[40]  Zhe L. Lin,et al.  Top-Down Neural Attention by Excitation Backprop , 2016, International Journal of Computer Vision.

[41]  D. Winter Kinematic and kinetic patterns in human gait: Variability and compensating effects , 1984 .

[42]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[43]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[44]  Alexander Binder,et al.  Explaining nonlinear classification decisions with deep Taylor decomposition , 2015, Pattern Recognit..

[45]  Angkoon Phinyomark,et al.  Analysis of Big Data in Gait Biomechanics: Current Trends and Future Directions , 2017, Journal of Medical and Biological Engineering.

[46]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[47]  Gunnar Rätsch,et al.  The Feature Importance Ranking Measure , 2009, ECML/PKDD.

[48]  Kushang V Patel,et al.  Predicting late-life disability and death by the rate of decline in physical performance measures. , 2012, Age and ageing.

[49]  Klaus-Robert Müller,et al.  A public dataset of overground walking kinetics and full-body kinematics in healthy adult individuals , 2019 .

[50]  Jacqueline Raymond,et al.  1000 Norms Project: protocol of a cross-sectional study cataloging human variation. , 2016, Physiotherapy.

[51]  D. Janssen,et al.  Recognition of Emotions in Gait Patterns by Means of Artificial Neural Nets , 2008 .

[52]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[53]  Alexandre Tkatchenko,et al.  Quantum-chemical insights from deep tensor neural networks , 2016, Nature Communications.

[54]  Karl M Newell,et al.  Intra-individual gait patterns across different time-scales as revealed by means of a supervised learning model using kernel-based discriminant regression , 2017, PloS one.

[55]  Joana Figueiredo,et al.  Automatic recognition of gait patterns in human motor disorders using machine learning: A review. , 2018, Medical engineering & physics.

[56]  Alexander Binder,et al.  On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , 2015, PloS one.

[57]  S. Studenski,et al.  Gait speed and survival in older adults. , 2011, JAMA.

[58]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[59]  Gunnar Rätsch,et al.  SVM2Motif—Reconstructing Overlapping DNA Sequence Motifs by Mimicking an SVM Predictor , 2015, PloS one.

[60]  Karl M Newell,et al.  Diagnosing fatigue in gait patterns by support vector machines and self-organizing maps. , 2011, Human movement science.

[61]  Benno M Nigg,et al.  Marker-based classification of young–elderly gait pattern differences via direct PCA feature extraction and SVMs , 2013, Computer methods in biomechanics and biomedical engineering.

[62]  Aurélien Garivier,et al.  On the Complexity of Best-Arm Identification in Multi-Armed Bandit Models , 2014, J. Mach. Learn. Res..

[63]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[64]  D. Elliott,et al.  The effects of targeting on the ground reaction forces during level walking , 1993 .

[65]  Klaus-Robert Müller,et al.  "What is relevant in a text document?": An interpretable machine learning approach , 2016, PloS one.

[66]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[67]  Herman Buschke,et al.  Abnormality of gait as a predictor of non-Alzheimer's dementia. , 2002, The New England journal of medicine.

[68]  Thompson Sarkodie-Gyan,et al.  Automatic classification of pathological gait patterns using ground reaction forces and machine learning algorithms , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[69]  S. Studenski,et al.  Physical Performance Measures in the Clinical Setting , 2003, Journal of the American Geriatrics Society.

[70]  W I Schöllhorn,et al.  Applications of artificial neural nets in clinical biomechanics. , 2004, Clinical biomechanics.

[71]  Roee Holtzer,et al.  Mobility Stress Test Approach to Predicting Frailty, Disability, and Mortality in High‐Functioning Older Adults , 2012, Journal of the American Geriatrics Society.

[72]  Wojciech Samek,et al.  Methods for interpreting and understanding deep neural networks , 2017, Digit. Signal Process..

[73]  Max Welling,et al.  A New Method to Visualize Deep Neural Networks , 2016, ArXiv.

[74]  Stefan Haufe,et al.  On the interpretation of weight vectors of linear models in multivariate neuroimaging , 2014, NeuroImage.

[75]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[76]  Fenglin Liu,et al.  Parkinson's disease classification using gait analysis via deterministic learning , 2016, Neuroscience Letters.

[77]  Jeffrey M. Hausdorff,et al.  Marked alterations in the gait timing and rhythmicity of patients with de novo Parkinson's disease , 2006, The European journal of neuroscience.

[78]  A. Steere,et al.  Dysregulation of CD4+CD25(high) T cells in the synovial fluid of patients with antibiotic-refractory Lyme arthritis. , 2013, Arthritis and rheumatism.

[79]  Bastiaan R. Bloem,et al.  Prevalence and Burden of Gait Disorders in Elderly Men and Women Aged 60–97 Years: A Population-Based Study , 2013, PloS one.

[80]  G M Earhart,et al.  Which measures of physical function and motor impairment best predict quality of life in Parkinson's disease? , 2011, Parkinsonism & related disorders.

[81]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[82]  B M Nigg,et al.  Identification of individual walking patterns using time discrete and time continuous data sets. , 2002, Gait & posture.

[83]  S C Wearing,et al.  The effect of visual targeting on ground reaction force and temporospatial parameters of gait. , 2000, Clinical biomechanics.

[84]  Jill Seale Gait in persons with chronic stroke: An investigation of overall gait and quality of life , 2010 .