On the Understanding and Interpretation of Machine Learning Predictions in Clinical Gait Analysis Using Explainable Artificial Intelligence

Systems incorporating Artificial Intelligence (AI) and machine learning (ML) techniques are increasingly used to guide decision-making in the healthcare sector. While AI-based systems provide powerful and promising results with regard to their classification and prediction accuracy (e.g., in differentiating between different disorders in human gait), most share a central limitation, namely their black-box character. Understanding which features classification models learn, whether they are meaningful and consequently whether their decisions are trustworthy is difficult and often impossible to comprehend. This severely hampers their applicability as decisionsupport systems in clinical practice. There is a strong need for AI-based systems to provide transparency and justification of predictions, which are necessary also for ethical and legal compliance. As a consequence, in recent years the field of explainable AI (XAI) has gained increasing importance. XAI focuses on the development of methods that enhance transparency and interpretability of complex ML models, such as Deep (Convolutional) Neural Networks. The primary aim of this article is to investigate whether XAI methods can enhance transparency, explainability and interpretability of predictions in automated clinical gait classification. We utilize a dataset comprising bilateral three-dimensional ground reaction force measurements from 132 patients with different lower-body gait disorders and 62 healthy controls. In our experiments, 1 ar X iv :1 91 2. 07 73 7v 1 [ cs .L G ] 1 6 D ec 2 01 9 Horst and Slijepcevic et al. Explainable AI in Clinical Gait Analysis we included several gait classification tasks, employed a representative set of classification methods, and a well-established XAI method – Layer-wise Relevance Propagation (LRP) – to explain decisions at the signal (input) level. The classification results are analyzed, compared and interpreted in terms of classification accuracy and relevance of input values for specific decisions. The decomposed input relevance information are evaluated from a statistical (using Statistical Parameter Mapping) and clinical (by an expert) viewpoint. There are three dimensions in our comparison: (i) different classification tasks, (ii) different classification methods, and (iii) data normalization. The presented approach exemplifies how XAI can be used to understand and interpret state-of-the-art ML models trained for gait classification tasks, and shows that the features that are considered relevant for machine learning models can be attributed to meaningful and clinically relevant biomechanical gait characteristics.