What is Unique in Individual Gait Patterns? Understanding and Interpreting Deep Learning in Gait Analysis

Machine learning (ML) techniques such as (deep) artificial neural networks (DNN) are solving very successfully a plethora of tasks and provide new predictive models for complex physical, chemical, biological and social systems. However, in most cases this comes with the disadvantage of acting as a black box, rarely providing information about what made them arrive at a particular prediction. This black box aspect of ML techniques can be problematic especially in medical diagnoses, so far hampering a clinical acceptance. The present paper studies the uniqueness of individual gait patterns in clinical biomechanics using DNNs. By attributing portions of the model predictions back to the input variables (ground reaction forces and full-body joint angles), the Layer-Wise Relevance Propagation (LRP) technique reliably demonstrates which variables at what time windows of the gait cycle are most relevant for the characterisation of gait patterns from a certain individual. By measuring the timeresolved contribution of each input variable to the prediction of ML techniques such as DNNs, our method describes the first general framework that enables to understand and interpret non-linear ML methods in (biomechanical) gait analysis and thereby supplies a powerful tool for analysis, diagnosis and treatment of human gait.

[1]  Alexander Binder,et al.  Analyzing Classifiers: Fisher Vectors and Deep Neural Networks , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Wolfgang I. Schöllhorn,et al.  One-year persistence of individual gait patterns identified in a follow-up study - A call for individualised diagnose and therapy. , 2017, Gait & posture.

[4]  Reed Ferber,et al.  Biomechanical Deviations During Level Walking Associated With Knee Osteoarthritis: A Systematic Review and Meta‐Analysis , 2013, Arthritis care & research.

[5]  Stefan Haufe,et al.  On the interpretation of weight vectors of linear models in multivariate neuroimaging , 2014, NeuroImage.

[6]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[7]  Karl M Newell,et al.  Diagnosing fatigue in gait patterns by support vector machines and self-organizing maps. , 2011, Human movement science.

[8]  Bastiaan R. Bloem,et al.  Prevalence and Burden of Gait Disorders in Elderly Men and Women Aged 60–97 Years: A Population-Based Study , 2013, PloS one.

[9]  Peter Federolf,et al.  A holistic approach to study the temporal variability in gait. , 2012, Journal of biomechanics.

[10]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[11]  Meg E. Morris,et al.  Classification of Gait Disorders Following Traumatic Brain Injury , 2015, The Journal of head trauma rehabilitation.

[12]  M. Gevrey,et al.  Review and comparison of methods to study the contribution of variables in artificial neural network models , 2003 .

[13]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[14]  Herman Buschke,et al.  Abnormality of gait as a predictor of non-Alzheimer's dementia. , 2002, The New England journal of medicine.

[15]  Thompson Sarkodie-Gyan,et al.  Automatic classification of pathological gait patterns using ground reaction forces and machine learning algorithms , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[16]  G M Earhart,et al.  Which measures of physical function and motor impairment best predict quality of life in Parkinson's disease? , 2011, Parkinsonism & related disorders.

[17]  Benno M Nigg,et al.  Marker-based classification of young–elderly gait pattern differences via direct PCA feature extraction and SVMs , 2013, Computer methods in biomechanics and biomedical engineering.

[18]  Subashan Perera,et al.  Improvements in Speed-Based Gait Classifications Are Meaningful , 2007, Stroke.

[19]  Benedicte Vanwanseele,et al.  Effect of children's shoes on gait: a systematic review and meta-analysis , 2011, Journal of foot and ankle research.

[20]  S. Studenski,et al.  Physical Performance Measures in the Clinical Setting , 2003, Journal of the American Geriatrics Society.

[21]  Fenglin Liu,et al.  Parkinson's disease classification using gait analysis via deterministic learning , 2016, Neuroscience Letters.

[22]  Jeffrey M. Hausdorff,et al.  Marked alterations in the gait timing and rhythmicity of patients with de novo Parkinson's disease , 2006, The European journal of neuroscience.

[23]  W I Schöllhorn,et al.  Applications of artificial neural nets in clinical biomechanics. , 2004, Clinical biomechanics.

[24]  Jeffrey Kaye,et al.  The trajectory of gait speed preceding mild cognitive impairment. , 2010, Archives of neurology.

[25]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[26]  Heekuck Oh,et al.  Neural Networks for Pattern Recognition , 1993, Adv. Comput..

[27]  B M Nigg,et al.  Identification of individual walking patterns using time discrete and time continuous data sets. , 2002, Gait & posture.

[28]  Motoaki Kawanabe,et al.  How to Explain Individual Classification Decisions , 2009, J. Mach. Learn. Res..

[29]  S C Wearing,et al.  The effect of visual targeting on ground reaction force and temporospatial parameters of gait. , 2000, Clinical biomechanics.

[30]  Jill Seale Gait in persons with chronic stroke: An investigation of overall gait and quality of life , 2010 .

[31]  R Begg,et al.  A machine learning approach for automated recognition of movement patterns using basic, kinetic and kinematic gait data. , 2005, Journal of biomechanics.

[32]  Klaus P. Ebmeier,et al.  What can gait tell us about dementia? Review of epidemiological and neuropsychological evidence. , 2017, Gait & posture.

[33]  Joana Figueiredo,et al.  Automatic recognition of gait patterns in human motor disorders using machine learning: A review. , 2018, Medical engineering & physics.

[34]  Alexander Binder,et al.  On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation , 2015, PloS one.

[35]  Gunnar Rätsch,et al.  SVM2Motif—Reconstructing Overlapping DNA Sequence Motifs by Mimicking an SVM Predictor , 2015, PloS one.

[36]  Hermann Schwameder,et al.  Computer aided analysis of gait patterns in patients with acute anterior cruciate ligament injury. , 2016, Clinical biomechanics.

[37]  Cecilia Fagerström,et al.  Mobility, functional ability and health-related quality of life among people of 60 years or older , 2010, Aging clinical and experimental research.

[38]  T Chau,et al.  A review of analytical techniques for gait data. Part 1: Fuzzy, statistical and fractal methods. , 2001, Gait & posture.

[39]  Gunnar Rätsch,et al.  The Feature Importance Ranking Measure , 2009, ECML/PKDD.

[40]  Kushang V Patel,et al.  Predicting late-life disability and death by the rate of decline in physical performance measures. , 2012, Age and ageing.

[41]  R. Lipton,et al.  Epidemiology of Gait Disorders in Community‐Residing Older Adults , 2006, Journal of the American Geriatrics Society.

[42]  Angkoon Phinyomark,et al.  Analysis of Big Data in Gait Biomechanics: Current Trends and Future Directions , 2017, Journal of Medical and Biological Engineering.

[43]  Benno M. Nigg,et al.  A public dataset of overground walking kinetics and lower-body kinematics in healthy adult individuals on different days , 2019 .

[44]  A. Atiya,et al.  Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond , 2005, IEEE Transactions on Neural Networks.

[45]  Alexander Binder,et al.  Understanding and Comparing Deep Neural Networks for Age and Gender Classification , 2017, 2017 IEEE International Conference on Computer Vision Workshops (ICCVW).

[46]  Klaus-Robert Müller,et al.  Explaining the unique nature of individual gait patterns with deep learning , 2018, Scientific Reports.

[47]  Klaus-Robert Müller,et al.  Interpretable deep neural networks for single-trial EEG classification , 2016, Journal of Neuroscience Methods.

[48]  Klaus-Robert Müller,et al.  A public dataset of overground walking kinetics and full-body kinematics in healthy adult individuals , 2019 .

[49]  Chih-Jen Lin,et al.  A Practical Guide to Support Vector Classication , 2008 .

[50]  Meg E Morris,et al.  Determinants of health-related quality of life in Parkinson's disease: a systematic review. , 2011, Parkinsonism & related disorders.

[51]  Roee Holtzer,et al.  Mobility Stress Test Approach to Predicting Frailty, Disability, and Mortality in High‐Functioning Older Adults , 2012, Journal of the American Geriatrics Society.

[52]  Wojciech Samek,et al.  Methods for interpreting and understanding deep neural networks , 2017, Digit. Signal Process..

[53]  Klaus-Robert Müller,et al.  "What is relevant in a text document?": An interpretable machine learning approach , 2016, PloS one.

[54]  Jacqueline Raymond,et al.  1000 Norms Project: protocol of a cross-sectional study cataloging human variation. , 2016, Physiotherapy.

[55]  D. Janssen,et al.  Recognition of Emotions in Gait Patterns by Means of Artificial Neural Nets , 2008 .

[56]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[57]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[58]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[59]  Andrew Zisserman,et al.  Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps , 2013, ICLR.

[60]  D. Elliott,et al.  The effects of targeting on the ground reaction forces during level walking , 1993 .

[61]  Andrea Vedaldi,et al.  Interpretable Explanations of Black Boxes by Meaningful Perturbation , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[62]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[63]  Jeffrey M. Hausdorff,et al.  Classification of gait disturbances: Distinguishing between continuous and episodic changes , 2013, Movement disorders : official journal of the Movement Disorder Society.

[64]  T Chau,et al.  A review of analytical techniques for gait data. Part 2: neural network and wavelet methods. , 2001, Gait & posture.

[65]  C. Powers,et al.  Quality Indicators for the Management and Prevention of Falls and Mobility Problems in Vulnerable Elders , 2001, Annals of Internal Medicine.

[66]  Alexander Binder,et al.  The LRP Toolbox for Artificial Neural Networks , 2016, J. Mach. Learn. Res..

[67]  Ralf Mikut,et al.  Automated feature assessment in instrumented gait analysis. , 2006, Gait & posture.

[68]  Carole A. Tucker,et al.  Measuring Walking: A Handbook of Clinical Gait Analysis , 2014 .

[69]  S. Studenski,et al.  Gait speed and survival in older adults. , 2011, JAMA.

[70]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[71]  Wolfgang I. Schöllhorn,et al.  A public dataset of overground walking kinetics in healthy individuals , 2018 .

[72]  Caterina Pesce,et al.  Health and Quality of Life Perception in Older Adults: The Joint Role of Cognitive Efficiency and Functional Mobility , 2015, International journal of environmental research and public health.

[73]  Benno M. Nigg,et al.  Daily changes of individual gait patterns identified by means of support vector machines. , 2016, Gait & posture.

[74]  Alexandre Tkatchenko,et al.  Quantum-chemical insights from deep tensor neural networks , 2016, Nature Communications.

[75]  Karl M Newell,et al.  Intra-individual gait patterns across different time-scales as revealed by means of a supervised learning model using kernel-based discriminant regression , 2017, PloS one.

[76]  R. Durbin,et al.  Using probabilistic estimation of expression residuals (PEER) to obtain increased power and interpretability of gene expression analyses , 2012, Nature Protocols.

[77]  Alexander Binder,et al.  Evaluating the Visualization of What a Deep Neural Network Has Learned , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[78]  Alexander Binder,et al.  Explaining nonlinear classification decisions with deep Taylor decomposition , 2015, Pattern Recognit..