Machine learning in secondary progressive multiple sclerosis: an improved predictive model for short-term disability progression

Background Enhanced prediction of progression in secondary progressive multiple sclerosis (SPMS) could improve clinical trial design. Machine learning (ML) algorithms are methods for training predictive models with minimal human intervention. Objective To evaluate individual and ensemble model performance built using decision tree (DT)-based algorithms compared to logistic regression (LR) and support vector machines (SVMs) for predicting SPMS disability progression. Methods SPMS participants (n = 485) enrolled in a 2-year placebo-controlled (negative) trial assessing the efficacy of MBP8298 were classified as progressors if a 6-month sustained increase in Expanded Disability Status Scale (EDSS) (≥1.0 or ≥0.5 for a baseline of ≤5.5 or ≥6.0 respectively) was observed. Variables included EDSS, Multiple Sclerosis Functional Composite component scores, T2 lesion volume, brain parenchymal fraction, disease duration, age, and sex. Area under the receiver operating characteristic curve (AUC) was the primary outcome for model evaluation. Results Three DT-based models had greater AUCs (61.8%, 60.7%, and 60.2%) than independent and ensemble SVM (52.4% and 51.0%) and LR (49.5% and 51.1%). Conclusion SPMS disability progression was best predicted by non-parametric ML. If confirmed, ML could select those with highest progression risk for inclusion in SPMS trial cohorts and reduce the number of low-risk individuals exposed to experimental therapies.

[1]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[2]  O. Ciccarelli,et al.  Predicting outcome in clinically isolated syndrome using machine learning , 2014, NeuroImage: Clinical.

[3]  C Zoccali,et al.  Linear and logistic regression analysis. , 2008, Kidney international.

[4]  R. Polikar,et al.  Ensemble based systems in decision making , 2006, IEEE Circuits and Systems Magazine.

[5]  Pierre Grammond,et al.  Defining secondary progressive multiple sclerosis. , 2016, Brain : a journal of neurology.

[6]  C. Brodley,et al.  Exploration of machine learning techniques in predicting multiple sclerosis disease course , 2017, PloS one.

[7]  O. Ciccarelli,et al.  Brain atrophy and lesion load predict long term disability in multiple sclerosis , 2013, Journal of Neurology, Neurosurgery & Psychiatry.

[8]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[9]  Mark Mühlau,et al.  Predicting conversion from clinically isolated syndrome to multiple sclerosis–An imaging-based machine learning approach , 2018, NeuroImage: Clinical.

[10]  Sabine Van Huffel,et al.  Machine Learning Approach for Classifying Multiple Sclerosis Courses by Combining Clinical Data with Lesion Loads and Magnetic Resonance Metabolic Features , 2017, Front. Neurosci..

[11]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[12]  M. Pepe,et al.  Comparing the predictive values of diagnostic tests: sample size and analysis for paired study designs , 2006, Clinical trials.

[13]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[14]  Josué Luiz Dalboni da Rocha,et al.  Characterization of relapsing-remitting multiple sclerosis patients using support vector machine classifications of functional and diffusion MRI data , 2018, NeuroImage: Clinical.

[15]  C. Florkowski Sensitivity, specificity, receiver-operating characteristic (ROC) curves and likelihood ratios: communicating the performance of diagnostic tests. , 2008, The Clinical biochemist. Reviews.

[16]  Doina Precup,et al.  Prediction of Disease Progression in Multiple Sclerosis Patients using Deep Learning Analysis of MRI Data , 2019, MIDL.

[17]  Ludwig Kappos,et al.  MRI-based prediction of conversion from clinically isolated syndrome to clinically definite multiple sclerosis using SVM and lesion geometry , 2018, Brain Imaging and Behavior.

[18]  A. Traboulsee,et al.  A phase III study evaluating the efficacy and safety of MBP8298 in secondary progressive MS , 2011, Neurology.

[19]  S. Reingold,et al.  The Multiple Sclerosis Functional Composite measure (MSFC): an integrated approach to MS clinical outcome assessment , 1999, Multiple sclerosis.

[20]  Xu Sun,et al.  Fast Implementation of DeLong’s Algorithm for Comparing the Areas Under Correlated Receiver Operating Characteristic Curves , 2014, IEEE Signal Processing Letters.

[21]  B. Giraudeau,et al.  Unbalanced rather than balanced randomized controlled trials are more often positive in favor of the new treatment: an exposed and nonexposed study. , 2015, Journal of clinical epidemiology.

[22]  L. Koski,et al.  Combined structural and functional patterns discriminating upper limb motor disability in multiple sclerosis using multivariate approaches , 2016, Brain Imaging and Behavior.

[23]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[24]  Donald A. Berry,et al.  Sequential Statistical Methods , 2015 .

[25]  Vikas Singh,et al.  Imaging-based enrichment criteria using deep learning algorithms for efficient clinical trials in mild cognitive impairment , 2015, Alzheimer's & Dementia.

[26]  J. Ball,et al.  Statistics review 12: Survival analysis , 2004, Critical care.

[27]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[28]  A. Trajman,et al.  McNemar χ2 test revisited: comparing sensitivity and specificity of diagnostic examinations , 2008, Scandinavian journal of clinical and laboratory investigation.

[29]  Ross Bettinger,et al.  Cost-Sensitive Classifier Selection Using the ROC Convex Hull Method , 2022 .

[30]  Vikas Singh,et al.  Randomized Deep Learning Methods for Clinical Trial Enrichment and Design in Alzheimer's Disease , 2017, Deep Learning for Medical Image Analysis.