Osteoporotic hip fracture prediction from risk factors available in administrative claims data – A machine learning approach

Objective Hip fractures are among the most frequently occurring fragility fractures in older adults, associated with a loss of quality of life, high mortality, and high use of healthcare resources. The aim was to apply the superlearner method to predict osteoporotic hip fractures using administrative claims data and to compare its performance to established methods. Methods We devided claims data of 288,086 individuals aged 65 years and older without care level into a training (80%) and a validation set (20%). Subsequently, we trained a superlearner algorithm that considered both regression and machine learning algorithms (e.g., support vector machines, RUSBoost) on a large set of clinical risk factors. Mean squared error and measures of discrimination and calibration were employed to assess prediction performance. Results All algorithms used in the analysis showed similar performance with an AUC ranging from 0.66 to 0.72 in the training and 0.65 to 0.70 in the validation set. Superlearner showed good discrimination in the training set but poorer discrimination and calibration in the validation set. Conclusions The superlearner achieved similar predictive performance compared to the individual algorithms included. Nevertheless, in the presence of non-linearity and complex interactions, this method might be a flexible alternative to be considered for risk prediction in large datasets.

[1]  J. Kai,et al.  Can machine-learning improve cardiovascular risk prediction using routine clinical data? , 2017, PloS one.

[2]  Christian Kruse,et al.  Machine Learning Principles Can Improve Hip Fracture Prediction , 2017, Calcified Tissue International.

[3]  C. Becker,et al.  Development of a risk assessment tool for osteoporotic fracture prevention: A claims data approach. , 2018, Bone.

[4]  Elad Eban,et al.  Scalable Learning of Non-Decomposable Objectives , 2016, AISTATS.

[5]  Taghi M. Khoshgoftaar,et al.  RUSBoost: A Hybrid Approach to Alleviating Class Imbalance , 2010, IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans.

[6]  Fenglong Ma,et al.  Dipole: Diagnosis Prediction in Healthcare via Attention-based Bidirectional Recurrent Neural Networks , 2017, KDD.

[7]  D. Hosmer,et al.  Applied Logistic Regression , 1991 .

[8]  O. Johnell,et al.  FRAX™ and the assessment of fracture probability in men and women from the UK , 2008, Osteoporosis International.

[9]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[10]  S. Rose Mortality risk score prediction in an elderly population using machine learning. , 2013, American journal of epidemiology.

[11]  Mark J. van der Laan,et al.  Super Learner In Prediction , 2010 .

[12]  C. Becker,et al.  Risk of osteoporotic fractures following stroke in older persons , 2015, Osteoporosis International.

[13]  Taghi M. Khoshgoftaar,et al.  RUSBoost: Improving classification performance when training data is skewed , 2008, 2008 19th International Conference on Pattern Recognition.

[14]  C. Cooper,et al.  Osteoporosis in the European Union: medical management, epidemiology and economic burden , 2013, Archives of Osteoporosis.

[15]  Karthik Ramasubramanian,et al.  Machine Learning Theory and Practices , 2017 .

[16]  Jasmin Häcker,et al.  Los(T) in long-term care: empirical evidence from German data 2000-2009. , 2012, Health economics.

[17]  S. Cummings,et al.  A comparison of prediction models for fractures in older women: is more better? , 2009, Archives of internal medicine.

[18]  M. Cecchini,et al.  Ultrastructural Characterization of the Lower Motor System in a Mouse Model of Krabbe Disease , 2016, Scientific Reports.

[19]  F. Cappuccio,et al.  Predicting risk of osteoporotic fracture in men and women in England and Wales : prospective derivation and validation of QFractureScores , 2017 .

[20]  Nishant Kumar,et al.  Using big data to enhance the bosch production line performance: A Kaggle challenge , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[21]  Mark van der Laan,et al.  Use of a machine learning framework to predict substance use disorder treatment success , 2017, PloS one.

[22]  Geoffrey E. Hinton,et al.  A comparison of statistical learning methods on the Gusto database. , 1998, Statistics in medicine.

[23]  C. Cooper,et al.  A systematic review of hip fracture incidence and probability of fracture worldwide , 2012, Osteoporosis International.

[24]  Marco Saerens,et al.  Adjusting the Outputs of a Classifier to New a Priori Probabilities: A Simple Procedure , 2002, Neural Computation.

[25]  Carol Coupland,et al.  Derivation and validation of updated QFracture algorithm to predict risk of osteoporotic fracture in primary care in the United Kingdom: prospective open cohort study , 2012, BMJ : British Medical Journal.

[26]  Jie Ma,et al.  A systematic review shows no performance benefit of machine learning over logistic regression for clinical prediction models. , 2019, Journal of clinical epidemiology.

[27]  M. Beyer,et al.  [Garbage in - garbage out? Validity of coded diagnoses from GP claims records]. , 2009, Gesundheitswesen (Bundesverband der Arzte des Offentlichen Gesundheitsdienstes (Germany)).

[28]  C. Cooper,et al.  Use of Inhaled Corticosteroids and Risk of Fractures , 2001, Journal of bone and mineral research : the official journal of the American Society for Bone and Mineral Research.

[29]  J. Kelsey,et al.  Risk Factors for Hip Fracture in Men , 1997 .

[30]  C. Cooper,et al.  Osteoporosis: burden, health care provision and opportunities in the EU , 2011, Archives of osteoporosis.

[31]  Li Li,et al.  Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records , 2016, Scientific Reports.

[32]  Stan Matwin,et al.  Addressing the Curse of Imbalanced Training Sets: One-Sided Selection , 1997, ICML.

[33]  M. Bech,et al.  Comparison of different screening tools (FRAX®, OST, ORAI, OSIRIS, SCORE and age alone) to identify women with increased risk of fracture. A population-based prospective study. , 2013, Bone.

[34]  Mark J van der Laan,et al.  Super Learning: An Application to the Prediction of HIV-1 Drug Resistance , 2007, Statistical applications in genetics and molecular biology.

[35]  R. Tamblyn,et al.  Validation of diagnostic codes within medical services claims. , 2004, Journal of clinical epidemiology.

[36]  W. O'Fallon,et al.  Fracture Incidence in Olmsted County, Minnesota: Comparison of Urban with Rural Rates and Changes in Urban Rates Over time , 1999, Osteoporosis International.

[37]  Foster J. Provost,et al.  Learning When Training Data are Costly: The Effect of Class Distribution on Tree Induction , 2003, J. Artif. Intell. Res..

[38]  F. Gerlach,et al.  Garbage in Garbage out? Validitt von Abrechnungsdiagnosen in hausrztlichen Praxen , 2009 .

[39]  Vasile Palade,et al.  Efficient resampling methods for training support vector machines with imbalanced datasets , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[40]  Elad Eban,et al.  Large-scale Learning With Global Non-Decomposable Objectives , 2016, ArXiv.

[41]  Eugene McCloskey,et al.  FRAX and its applications to clinical practice. , 2009, Bone.

[42]  J. Zeidler,et al.  Access, use, and challenges of claims data analyses in Germany , 2017, The European Journal of Health Economics.

[43]  L. Lix,et al.  Comparison between various fracture risk assessment tools , 2013, Osteoporosis International.

[44]  O. Johnell,et al.  Life-style and different fracture prevalence: A cross-sectional comparative population-based study , 1993, Calcified Tissue International.

[45]  Alessandro Tredicucci,et al.  Corrigendum: Universal lineshapes at the crossover between weak and strong critical coupling in Fano-resonant coupled oscillators , 2016, Scientific Reports.

[46]  Sherri Rose,et al.  Prediction of absolute risk of acute graft-versus-host disease following hematopoietic cell transplantation , 2018, PloS one.

[47]  J. Pfeilschifter CME: DVO-Leitlinie 2006 zur Prophylaxe, Diagnostik und Therapie der Osteoporose bei Frauen ab der Menopause, bei Männern ab dem 60. Lebensjahr - Kurzfassung , 2006 .

[48]  Gustavo E. A. P. A. Batista,et al.  A study of the behavior of several methods for balancing machine learning training data , 2004, SKDD.

[49]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[50]  Rishi Sikka,et al.  Estimating medication persistency using administrative claims data. , 2005, The American journal of managed care.

[51]  Jennifer Hill,et al.  Automated versus Do-It-Yourself Methods for Causal Inference: Lessons Learned from a Data Analysis Competition , 2017, Statistical Science.

[52]  Ewout W Steyerberg,et al.  Modern modelling techniques are data hungry: a simulation study for predicting dichotomous endpoints , 2014, BMC Medical Research Methodology.

[53]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[54]  M. J. van der Laan,et al.  Mortality prediction in intensive care units with the Super ICU Learner Algorithm (SICULA): a population-based study. , 2015, The Lancet. Respiratory medicine.

[55]  Simon Fong,et al.  An Application of Oversampling, Undersampling, Bagging and Boosting in Handling Imbalanced Datasets , 2013, DaEng.

[56]  Sabine Vanhuysse,et al.  Very High Resolution Object-Based Land Use–Land Cover Urban Classification Using Extreme Gradient Boosting , 2018, IEEE Geoscience and Remote Sensing Letters.