Application of Machine Learning to Improve Appropriateness of Treatment in an Orthopaedic Setting of Personalized Medicine

The rise of personalized medicine and its remarkable advancements have revealed new requirements for the availability of appropriate medical decision-making models. Computer science is an area that plays an essential role in the field of personalized medicine, where one of the goals is to provide algorithms and tools to extrapolate knowledge and improve the decision-support process. The minimum clinically important difference (MCID) is the smallest change in PROM scores that patients perceive as meaningful. Treatment that does not achieve the minimum level of improvement is considered inappropriate as well as a potential waste of resources. Using the MCID threshold to identify patients who fail to achieve the minimum change in PROM that results in a meaningful outcome may aid in pre-surgical shared decision-making. The decision tree algorithm is a method for extracting valuable information and providing further meaningful information to the domain expert that supports the decision-making. In the present study, different tools based on machine learning were developed. On the one hand, we compared three XGBoost models to predict the non-achievement of the MCID at six months post-operation in the SF-12 physical score. The prediction score threshold was set to 0.75 to provide three decision-making areas on the basis of the high confidence (HC) intervals; the minority class was re-balanced by weighting the positive class to penalize the loss function (XGBoost cost-sensitive), oversampling the minority class (XGBoost with SMOTE), and re-sampling the negative class (XGBoost with undersampling). On the other hand, we modeled the data through a decision tree (assessment tree), based on different complexity levels, to identify the hidden pattern and to provide a new way to understand possible relationships between the gathered features and the several outcomes. The results showed that all the proposed models were effective as binary classifiers, as they showed moderate predictive performance both regarding the minority or positive class (i.e., our targeted patients, those who will not benefit from surgery) and the negative class. The decision tree visualization can be exploited during the patient assessment status to better understand if those patients will benefit or not from the medical intervention. Both of these tools can come in handy for increasing knowledge about the patient’s psychophysical state and for creating an increasingly specialized assessment of the individual patient.

[1]  M. Austin,et al.  Should We Aim to Help Patients "Feel Better" or "Feel Good" After Total Hip Arthroplasty? Determining Factors Affecting the Achievement of the Minimal Clinically Important Difference and Patient Acceptable Symptom State. , 2022, The Journal of arthroplasty.

[2]  K. Camphausen,et al.  Bias and Class Imbalance in Oncologic Data—Towards Inclusive and Transferrable AI in Large Scale Oncology Data Sets , 2022, Cancers.

[3]  Qingyi Lin,et al.  A Comparison of XGBoost, Random Forest, and Nomograph for the Prediction of Disease Severity in Patients With COVID-19 Pneumonia: Implications of Cytokine and Immune Cell Profile , 2022, Frontiers in cellular and infection microbiology.

[4]  N. SooHoo,et al.  Development of a Machine Learning Algorithm for Prediction of Complications after Ankle Arthrodesis , 2022, Foot & ankle orthopaedics.

[5]  F. Cabitza,et al.  Global Interpretable Calibration Index, a New Metric to Estimate Machine Learning Models' Calibration , 2022, CD-MAKE.

[6]  J. Schwab,et al.  Development of machine learning algorithms to predict achievement of minimal clinically important difference for the KOOS‐PS following total knee arthroplasty , 2021, Journal of orthopaedic research : official publication of the Orthopaedic Research Society.

[7]  Siyuan Zhang,et al.  Machine learning algorithms do not outperform preoperative thresholds in predicting clinically meaningful improvements after total knee arthroplasty , 2021, Knee Surgery, Sports Traumatology, Arthroscopy.

[8]  Benjamin S. Glicksberg,et al.  Predictive Approaches for Acute Dialysis Requirement and Death in COVID-19 , 2021, Clinical journal of the American Society of Nephrology : CJASN.

[9]  Victoria López,et al.  Personalized Characterization of Emotional States in Patients with Bipolar Disorder , 2021, Mathematics.

[10]  Ramon Casanova,et al.  Age Differences in Estimating Physical Activity by Wrist Accelerometry Using Machine Learning , 2021, Sensors.

[11]  Tianbao Yang,et al.  Stochastic Optimization of Areas Under Precision-Recall Curves with Provable Convergence , 2021, NeurIPS.

[12]  I. Oksuz,et al.  Comparison of machine learning methods for prediction of osteoradionecrosis incidence in patients with head and neck cancer , 2021, The British journal of radiology.

[13]  Jacques Wainer,et al.  Nested cross-validation when selecting classifiers is overzealous for most practical applications , 2018, Expert Syst. Appl..

[14]  Yanxia Sun,et al.  Performance analysis of cost-sensitive learning methods with application to imbalanced medical data , 2021, Informatics in Medicine Unlocked.

[15]  Nam N. Nguyen,et al.  Comparison of Two Main Approaches for Handling Imbalanced Data in Churn Prediction Problem , 2021 .

[16]  Olivier Lézoray,et al.  Medical decision-making based on the exploration of a personalized medicine dataset , 2021 .

[17]  Abhiram Prasad,et al.  Predictive programmatic re-targeting to improve website conversion rates , 2021 .

[18]  C. Terwee,et al.  Patient‐reported outcome measures ( PROMs): making sense of individual PROM scores and changes in PROM scores over time , 2020, Nephrology.

[19]  L. Carreon,et al.  Applied Machine Learning for Spine Surgeons: Predicting Outcome for Patients Undergoing Treatment for Lumbar Disc Herniation Using PRO Data , 2020, Global spine journal.

[20]  N. Forkert,et al.  Machine Learning for Precision Medicine. , 2020, Genome.

[21]  Dean F. Sittig,et al.  Modeling physician variability to prioritize relevant medical record information , 2020, medRxiv.

[22]  N. Peters,et al.  Interpretable XGBoost Based Classification of 12-lead ECGs Applying Information Theory Measures From Neuroscience , 2020, 2020 Computing in Cardiology.

[23]  Shannon M. Lynch,et al.  A Rigorous Machine Learning Analysis Pipeline for Biomedical Binary Classification: Application in Pancreatic Cancer Nested Case-control Studies with Implications for Bias Assessments , 2020, ArXiv.

[24]  Gonzalo Martínez-Muñoz,et al.  A comparative analysis of gradient boosting algorithms , 2020, Artificial Intelligence Review.

[25]  Giorgio Visani,et al.  Metrics for Multi-Class Classification: an Overview , 2020, ArXiv.

[26]  Omar F. Khan,et al.  Use of Machine Learning and Artificial Intelligence to Drive Personalized Medicine Approaches for Spine Care. , 2020, World neurosurgery.

[27]  F. Cabitza,et al.  ALL YOU NEED IS HIGHER ACCURACY? ON THE QUEST FOR MINIMUM ACCEPTABLE ACCURACY FOR MEDICAL ARTIFICIAL INTELLIGENCE , 2020, Proceedings of the 12th International Conference on e-Health (EH2020).

[28]  Alex H. S. Harris,et al.  Can Machine Learning Methods Produce Accurate and Easy-to-Use Preoperative Prediction Models of One-Year Improvements in Pain and Functioning After Knee Arthroplasty? , 2020, The Journal of arthroplasty.

[29]  J. Schwab,et al.  Development of Machine Learning Algorithms to Predict Clinically Meaningful Improvement for the Patient-Reported Health State After Total Hip Arthroplasty. , 2020, The Journal of arthroplasty.

[30]  L. Feldman,et al.  Practical Guide to Assessment of Patient-Reported Outcomes. , 2020, JAMA surgery.

[31]  J. Malec,et al.  A Standard Method for Determining the Minimal Clinically Important Difference for Rehabilitation Measures. , 2020, Archives of physical medicine and rehabilitation.

[32]  P. Wischmeyer,et al.  Perioperative Nutrition Care of Orthopedic Surgery Patient , 2019, Techniques in orthopaedics.

[33]  I. Kashirina,et al.  Building Models for Predicting Mortality after Myocardial Infarction in Conditions of Unbalanced Classes, Including the Influence of Weather Conditions , 2020, DAMDID/RCDL.

[34]  D. Çelik,et al.  Minimal clinically important difference of commonly used hip-, knee-, foot-, and ankle-specific questionnaires: a systematic review. , 2019, Journal of clinical epidemiology.

[35]  Shenglong Li,et al.  Research on orthopedic auxiliary classification and prediction model based on XGBoost algorithm , 2019, Neural Computing and Applications.

[36]  Michael Gribskov,et al.  IRESpy: an XGBoost model for prediction of internal ribosome entry sites , 2019, BMC Bioinformatics.

[37]  M. Fontana,et al.  Can Machine Learning Algorithms Predict Which Patients Will Achieve Minimally Clinically Important Differences From Total Joint Arthroplasty? , 2019, Clinical Orthopaedics and Related Research.

[38]  V. Brodszky,et al.  Is the trend of increasing use of patient-reported outcome measures in medical device studies the sign of shift towards value-based purchasing in Europe? , 2019, The European Journal of Health Economics.

[39]  K. Bozic,et al.  Value-based Health Care: Moving Beyond "Minimum Clinically Important Difference" to a Tiered System of Evaluating Successful Clinical Outcomes. , 2019, Clinical orthopaedics and related research.

[40]  Victor E. Staartjes,et al.  Deep learning-based preoperative predictive analytics for patient-reported outcomes following lumbar discectomy: feasibility of center-specific modeling. , 2019, The spine journal : official journal of the North American Spine Society.

[41]  Hang Lei,et al.  Hyperparameter Optimization for Machine Learning Models Based on Bayesian Optimization , 2019 .

[42]  Reiner Leidl,et al.  Predicting patient-reported outcomes following hip and knee replacement surgery using supervised machine learning , 2019, BMC Medical Informatics and Decision Making.

[43]  C. Munthe,et al.  Person Centered Care and Personalized Medicine: Irreconcilable Opposites or Potential Companions? , 2019, Health Care Analysis.

[44]  M. Kool,et al.  Integrating Patient Perspectives into Personalized Medicine in Idiopathic Pulmonary Fibrosis , 2017, Front. Med..

[45]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[46]  R. Stolker,et al.  Obesity – a risk factor for postoperative complications in general surgery? , 2015, BMC Anesthesiology.

[47]  Ying LU,et al.  Decision tree methods: applications for classification and prediction , 2015, Shanghai archives of psychiatry.

[48]  J. Keilwagen,et al.  Area under Precision-Recall Curves for Weighted and Unweighted Data , 2014, PloS one.

[49]  J. Argenson,et al.  Obesity in orthopedics and trauma surgery. , 2014, Orthopaedics & traumatology, surgery & research : OTSR.

[50]  Andreas Holzinger,et al.  Trends in Interactive Knowledge Discovery for Personalized Medicine: Cognitive Science meets Machine Learning , 2014, IEEE Intell. Informatics Bull..

[51]  Carla E. Brodley,et al.  Class Imbalance, Redux , 2011, 2011 IEEE 11th International Conference on Data Mining.

[52]  A. Copay,et al.  Understanding the minimum clinically important difference: a review of concepts and methods. , 2007, The spine journal : official journal of the North American Spine Society.

[53]  O. Nafiu,et al.  Postoperative Complications in Obese and Nonobese Patients , 2007, World Journal of Surgery.

[54]  Yang Wang,et al.  Pattern discovery: a data driven approach to decision support , 2003, IEEE Trans. Syst. Man Cybern. Part C.

[55]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[56]  Syed Sibte Raza Abidi,et al.  Knowledge management in healthcare: towards 'knowledge-driven' decision-support services , 2001, Int. J. Medical Informatics.

[57]  Tony R. Martinez,et al.  Distribution-balanced stratified cross-validation for accuracy estimation , 2000, J. Exp. Theor. Artif. Intell..

[58]  G Gigerenzer,et al.  Reasoning the fast and frugal way: models of bounded rationality. , 1996, Psychological review.

[59]  David C. Hay,et al.  Data Model Patterns: Conventions of Thought , 1965 .