Visualizing Ubiquitously Sensed Measures of Motor Ability in Multiple Sclerosis

Sophisticated ubiquitous sensing systems are being used to measure motor ability in clinical settings. Intended to augment clinical decision-making, the interpretability of the machine-learning measurements underneath becomes critical to their use. We explore how visualization can support the interpretability of machine-learning measures through the case of Assess MS, a system to support the clinical assessment of Multiple Sclerosis. A substantial design challenge is to make visible the algorithm's decision-making process in a way that allows clinicians to integrate the algorithm's result into their own decision process. To this end, we present a series of design iterations that probe the challenges in supporting interpretability in a real-world system. The key contribution of this article is to illustrate that simply making visible the algorithmic decision-making process is not helpful in supporting clinicians in their own decision-making process. It disregards that people and algorithms make decisions in different ways. Instead, we propose that visualisation can provide context to algorithmic decision-making, rendering observable a range of internal workings of the algorithm from data quality issues to the web of relationships generated in the machine-learning process.

[1]  Lise Worthen-Chaudhari,et al.  Proof of Concept of the Ability of the Kinect to Quantify Upper Extremity Function in Dystrophinopathy , 2013, PLoS currents.

[2]  Advait Sarkar,et al.  Confidence, command, complexity: metamodels for structured interaction with machine intelligence , 2015, PPIG.

[3]  Desney S. Tan,et al.  Interactive optimization for steering machine classification , 2010, CHI.

[4]  Sheldon R Simon,et al.  Quantification of human motion: gait analysis-benefits and limitations to its application to clinical problems. , 2004, Journal of biomechanics.

[5]  Cathy Craig,et al.  Assessing and training standing balance in older adults: a novel approach using the 'Nintendo Wii' Balance Board. , 2011, Gait & posture.

[6]  Ana Tajadura-Jiménez,et al.  Motivating people with chronic pain to do physical activity: opportunities for technology design , 2014, CHI.

[7]  Marc Van Droogenbroeck,et al.  GAIMS: a powerful gait analysis system satisfying the constraints of clinical routine , 2013 .

[8]  Alan Borning,et al.  Fine-grained sharing of sensed physical activity: a value sensitive approach , 2013, UbiComp.

[9]  Ludwig Kappos,et al.  Quantifying Progression of Multiple Sclerosis via Classification of Depth Videos , 2014, MICCAI.

[10]  Helena M. Mentis,et al.  Learning to see the body: supporting instructional practices in laparoscopic surgical procedures , 2014, CHI.

[11]  B. Asher The Professional Vision , 1994 .

[12]  Kenneth Gersing,et al.  VisualDecisionLinc: A visual analytics approach for comparative effectiveness-based clinical decision support in psychiatry , 2012, J. Biomed. Informatics.

[13]  Dympna O'Sullivan,et al.  The Role of Explanations on Trust and Reliance in Clinical Decision Support Systems , 2015, 2015 International Conference on Healthcare Informatics.

[14]  Ratul Mahajan,et al.  CueT: human-guided fast and accurate network alarm triage , 2011, CHI.

[15]  Weng-Keen Wong,et al.  Why-oriented end-user debugging of naive Bayes text classification , 2011, ACM Trans. Interact. Intell. Syst..

[16]  Helena M. Mentis,et al.  Imaging the body: embodied vision in minimally invasive surgery , 2013, CHI.

[17]  Lisa Singh,et al.  G-PARE: A visual analytic tool for comparative analysis of uncertain graphs , 2011, 2011 IEEE Conference on Visual Analytics Science and Technology (VAST).

[18]  Johannes Gehrke,et al.  Accurate intelligible models with pairwise interactions , 2013, KDD.

[19]  Perry R. Cook,et al.  Human model evaluation in interactive supervised learning , 2011, CHI.

[20]  Anind K. Dey,et al.  Why and why not explanations improve the intelligibility of context-aware intelligent systems , 2009, CHI.

[21]  Kenton O'Hara,et al.  Assessing Multiple Sclerosis With Kinect: Designing Computer Vision Systems for Real-World Use , 2016, Hum. Comput. Interact..

[22]  Jeffrey A Cohen,et al.  Disability outcome measures in multiple sclerosis clinical trials: current status and future prospects , 2012, The Lancet Neurology.

[23]  Weng-Keen Wong,et al.  Principles of Explanatory Debugging to Personalize Interactive Machine Learning , 2015, IUI.

[24]  Tal Arbel,et al.  Rate of Agreement for Manual and Automated Techniques for Determination of New T2 Lesions in Children with Multiple Sclerosis and Acute Demyelination (P2.242) , 2014 .

[25]  Christian P. Kamm,et al.  Multiple Sclerosis: Current Knowledge and Future Outlook , 2014, European Neurology.

[26]  Alex Voss,et al.  'Repairing' the Machine: A Case Study of the Evaluation of Computer-Aided Detection Tools in Breast Screening , 2003, ECSCW.

[27]  Peter Kontschieder,et al.  Setwise Comparison: Consistent, Scalable, Continuum Labels for Computer Vision , 2016, CHI.

[28]  Wassim M. Haddad,et al.  A Microsoft Kinect-Based Point-of-Care Gait Assessment Framework for Multiple Sclerosis Patients , 2017, IEEE Journal of Biomedical and Health Informatics.

[29]  Thompson Sarkodie-Gyan,et al.  Automatic classification of pathological gait patterns using ground reaction forces and machine learning algorithms , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[30]  Anind K. Dey,et al.  Assessing demand for intelligibility in context-aware applications , 2009, UbiComp.

[31]  Bernard M. J. Uitdehaag,et al.  Usability and acceptability of ASSESS MS: a system to support the assessment of motor dysfunction in Multiple Sclerosis using depth-sensing computer vision , 2015 .

[32]  Dh Miller,et al.  Predicting clinical conversion to multiple sclerosis in patients with clinically isolated syndrome using machine learning techniques , 2012 .

[33]  Olivier Clatz,et al.  Spatial decision forests for MS lesion segmentation in multi-channel magnetic resonance images , 2011, NeuroImage.

[34]  René F. Kizilcec How Much Information?: Effects of Transparency on Trust in an Algorithmic Interface , 2016, CHI.

[35]  Taha Khan,et al.  A computer vision framework for finger-tapping evaluation in Parkinson's disease , 2014, Artif. Intell. Medicine.

[36]  Matjaz Gams,et al.  Combining human analysis and machine data mining to obtain credible data relations , 2014, Inf. Sci..

[37]  Lynne Baillie,et al.  A novel knee rehabilitation system for the home , 2014, CHI.

[38]  Desney S. Tan,et al.  Effective End-User Interaction with Machine Learning , 2011, AAAI.

[39]  John Riedl,et al.  Explaining collaborative filtering recommendations , 2000, CSCW '00.

[40]  Desney S. Tan,et al.  EnsembleMatrix: interactive visualization to support machine learning with multiple classifiers , 2009, CHI.

[41]  Johannes Gehrke,et al.  Intelligible Models for HealthCare: Predicting Pneumonia Risk and Hospital 30-day Readmission , 2015, KDD.

[42]  Sinziana Mazilu,et al.  GaitAssist: a daily-life support and training system for parkinson's disease patients with freezing of gait , 2014, CHI.

[43]  Ludwig Kappos,et al.  Usability and Acceptability of ASSESS MS: Assessment of Motor Dysfunction in Multiple Sclerosis Using Depth-Sensing Computer Vision , 2015, JMIR human factors.

[44]  B. Kaplan,et al.  Objectification and negotiation in interpreting clinical images: implications for computer-based patient records , 1995, Artif. Intell. Medicine.

[45]  Philip R. Cohen,et al.  Application of machine learning and numerical analysis to classify tremor in patients affected with essential tremor or Parkinson's disease , 2012 .

[46]  Thomas G. Dietterich,et al.  Interacting meaningfully with machine learning systems: Three experiments , 2009, Int. J. Hum. Comput. Stud..

[47]  J. Kurtzke Rating neurologic impairment in multiple sclerosis , 1983, Neurology.

[48]  Desney S. Tan,et al.  CueFlik: interactive concept learning in image search , 2008, CHI.

[49]  Jarke J. van Wijk,et al.  BaobabView: Interactive construction and analysis of decision trees , 2011, 2011 IEEE Conference on Visual Analytics Science and Technology (VAST).

[50]  Adele Cutler,et al.  An application of Random Forests to a genome-wide association dataset: Methodological considerations & new findings , 2010, BMC Genetics.