Prediction models for the mortality risk in chronic dialysis patients: a systematic review and independent external validation study

Objective In medicine, many more prediction models have been developed than are implemented or used in clinical practice. These models cannot be recommended for clinical use before external validity is established. Though various models to predict mortality in dialysis patients have been published, very few have been validated and none are used in routine clinical practice. The aim of the current study was to identify existing models for predicting mortality in dialysis patients through a review and subsequently to externally validate these models in the same large independent patient cohort, in order to assess and compare their predictive capacities. Methods A systematic review was performed following the preferred reporting items for systematic reviews and meta-analyses (PRISMA) guidelines. To account for missing data, multiple imputation was performed. The original prediction formulae were extracted from selected studies. The probability of death per model was calculated for each individual within the Netherlands Cooperative Study on the Adequacy of Dialysis (NECOSAD). The predictive performance of the models was assessed based on their discrimination and calibration. Results In total, 16 articles were included in the systematic review. External validation was performed in 1,943 dialysis patients from NECOSAD for a total of seven models. The models performed moderately to well in terms of discrimination, with C-statistics ranging from 0.710 (interquartile range 0.708–0.711) to 0.752 (interquartile range 0.750–0.753) for a time frame of 1 year. According to the calibration, most models overestimated the probability of death. Conclusion Overall, the performance of the models was poorer in the external validation than in the original population, affirming the importance of external validation. Floege et al’s models showed the highest predictive performance. The present study is a step forward in the use of a prediction model as a useful tool for nephrologists, using evidence-based medicine that combines individual clinical expertise, patients’ choices, and the best available external evidence.

[1]  A. Laupacis,et al.  Predicting the Risk of 1-Year Mortality in Incident Dialysis Patients: Accounting for Case-Mix Severity in Studies Using Administrative Data , 2011, Medical care.

[2]  F. Dekker,et al.  Improved Mortality Prediction in Dialysis Patients Using Specific Clinical and Laboratory Data , 2015, American Journal of Nephrology.

[3]  Yvonne Vergouwe,et al.  External validity of risk models: Use of benchmark values to disentangle a case-mix effect from incorrect coefficients. , 2010, American journal of epidemiology.

[4]  Y. Mori,et al.  Prognostic utility of plasma S100A12 levels to establish a novel scoring system for predicting mortality in maintenance hemodialysis patients: a two-year prospective observational study in Japan , 2013, BMC Nephrology.

[5]  Gary S Collins,et al.  Comparing risk prediction models , 2012, BMJ : British Medical Journal.

[6]  Karel G M Moons,et al.  A new framework to enhance the interpretation of external validation studies of clinical prediction models. , 2015, Journal of clinical epidemiology.

[7]  F. Dekker,et al.  Survival prognosis after the start of a renal replacement therapy in the Netherlands: a retrospective cohort study , 2013, BMC Nephrology.

[8]  David M Kent,et al.  Predicting mortality in incident dialysis patients: an analysis of the United Kingdom Renal Registry. , 2011, American journal of kidney diseases : the official journal of the National Kidney Foundation.

[9]  F. Dekker,et al.  When to initiate dialysis: effect of proposed US guidelines on survival , 2001, The Lancet.

[10]  Douglas G Altman,et al.  Combining estimates of interest in prognostic modelling studies after multiple imputation: current practice and guidelines , 2009, BMC medical research methodology.

[11]  Y. Vergouwe,et al.  Validation, updating and impact of clinical prediction rules: a review. , 2008, Journal of clinical epidemiology.

[12]  N. Powe,et al.  Early, intermediate, and long-term risk factors for mortality in incident dialysis patients: the Choices for Healthy Outcomes in Caring for ESRD (CHOICE) Study. , 2007, American journal of kidney diseases : the official journal of the National Kidney Foundation.

[13]  D. Sackett,et al.  Evidence based medicine: what it is and what it isn't , 1996, BMJ.

[14]  R. Ruthazer,et al.  Predicting six-month mortality for patients who are on maintenance hemodialysis. , 2010, Clinical journal of the American Society of Nephrology : CJASN.

[15]  N. Obuchowski,et al.  Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures , 2010, Epidemiology.

[16]  Yvonne Vergouwe,et al.  Prognosis and prognostic research: Developing a prognostic model , 2009, BMJ : British Medical Journal.

[17]  P. Royston,et al.  Prognosis and prognostic research: application and impact of prognostic models in clinical practice , 2009, BMJ : British Medical Journal.

[18]  Yang Qiu,et al.  'United States Renal Data System 2011 Annual Data Report: Atlas of chronic kidney disease & end-stage renal disease in the United States. , 2012, American journal of kidney diseases : the official journal of the National Kidney Foundation.

[19]  M. Woodward,et al.  Risk prediction models: II. External validation, model updating, and impact assessment , 2012, Heart.

[20]  Michael G Kenward,et al.  Multiple imputation: current perspectives , 2007, Statistical methods in medical research.

[21]  R. Foley,et al.  Advance prediction of early death in patients starting maintenance dialysis. , 1994, American journal of kidney diseases : the official journal of the National Kidney Foundation.

[22]  E. Steyerberg,et al.  Reporting and Methods in Clinical Prediction Research: A Systematic Review , 2012, PLoS medicine.

[23]  Yvonne Vergouwe,et al.  Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. , 2005, Journal of clinical epidemiology.

[24]  Anirudh Rao,et al.  UK Renal Registry 18th Annual Report (December 2015) Chapter 5: Survival and Causes of Death in UK Adult Patients on Renal Replacement Therapy in 2014: National and Centre-specific Analyses , 2016, Nephron.

[25]  Suguru Yamamoto,et al.  Risk Score to Predict 1-Year Mortality after Haemodialysis Initiation in Patients with Stage 5 Chronic Kidney Disease under Predialysis Nephrology Care , 2015, PloS one.

[26]  Y Vergouwe,et al.  Updating methods improved the performance of a clinical prediction model in new patients. , 2008, Journal of clinical epidemiology.

[27]  M. Clèries,et al.  Design and validation of a model to predict early mortality in haemodialysis patients. , 2008, Nephrology, dialysis, transplantation : official publication of the European Dialysis and Transplant Association - European Renal Association.

[28]  A. Zwinderman,et al.  The ERA-EDTA cohort study--comparison of methods to predict survival on renal replacement therapy. , 2006, Nephrology, dialysis, transplantation : official publication of the European Dialysis and Transplant Association - European Renal Association.

[29]  H. Chua,et al.  Predicting First-Year Mortality in Incident Dialysis Patients with End-Stage Renal Disease - The UREA5 Study , 2014, Blood Purification.

[30]  J. Ioannidis,et al.  The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration , 2009, BMJ : British Medical Journal.

[31]  Frits R Rosendaal,et al.  Cardiovascular and noncardiovascular mortality among patients starting dialysis. , 2009, JAMA.

[32]  M. Otero-López,et al.  Two prognostic scores for early mortality and their clinical applicability in elderly patients on haemodialysis: poor predictive success in individual patients. , 2012, Nefrologia : publicacion oficial de la Sociedad Espanola Nefrologia.

[33]  G. Collins,et al.  External validation of multivariable prediction models: a systematic review of methodological conduct and reporting , 2014, BMC Medical Research Methodology.

[34]  Gary S Collins,et al.  Sample size considerations for the external validation of a multivariable prognostic model: a resampling study , 2015, Statistics in medicine.

[35]  C. Chazot,et al.  The dynamics of prognostic indicators: toward earlier identification of dialysis patients with a high risk of dying. , 2013, Kidney international.

[36]  Sunil J Rao,et al.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis , 2003 .

[37]  L. Tang,et al.  Predicting One-Year Mortality in Peritoneal Dialysis Patients: An Analysis of the China Peritoneal Dialysis Registry , 2015, International journal of medical sciences.

[38]  Carmine Zoccali,et al.  Multiple imputation: dealing with missing data. , 2013, Nephrology, dialysis, transplantation : official publication of the European Dialysis and Transplant Association - European Renal Association.

[39]  Florian Kronenberg,et al.  Development and validation of a predictive mortality risk score from a European hemodialysis cohort , 2015, Kidney international.

[40]  Theo Stijnen,et al.  Using the outcome for imputation of missing predictor values was preferred. , 2006, Journal of clinical epidemiology.

[41]  Karel G M Moons,et al.  Prediction models for risk of developing type 2 diabetes: systematic literature search and independent external validation study , 2012, BMJ : British Medical Journal.

[42]  R. Steenkamp,et al.  Chapter 5 Survival and Cause of Death in UK Adult Patients on Renal Replacement Therapy in 2016: National and Centre-specific Analyses , 2018, Nephron.

[43]  A. Collins,et al.  Effect of comorbidity on the increased mortality associated with early initiation of dialysis. , 2005, American Journal of Kidney Diseases.

[44]  T. Hutchinson,et al.  Predicting survival in adults with end-stage renal disease: an age equivalence index. , 1982, Annals of internal medicine.

[45]  H C van Houwelingen,et al.  Validation, calibration, revision and combination of prognostic survival models. , 2000, Statistics in medicine.

[46]  J. Ioannidis,et al.  External validation of new risk prediction models is infrequent and reveals worse prognostic discrimination. , 2015, Journal of clinical epidemiology.

[47]  M. Woodward,et al.  Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker , 2012, Heart.

[48]  Yvonne Vergouwe,et al.  A simple method to adjust clinical prediction models to local circumstances , 2009, Canadian journal of anaesthesia = Journal canadien d'anesthesie.

[49]  P. Royston,et al.  External validation of a Cox prognostic model: principles and methods , 2013, BMC Medical Research Methodology.

[50]  Yvonne Vergouwe,et al.  Prognosis and prognostic research: validating a prognostic model , 2009, BMJ : British Medical Journal.

[51]  R Brian Haynes,et al.  Evidence based medicine: what it is and what it isn't. 1996. , 2007, Clinical orthopaedics and related research.

[52]  F. Zannad,et al.  Prognostic model for total mortality in patients with haemodialysis from the Assessments of Survival and Cardiovascular Events (AURORA) study , 2012, Journal of internal medicine.

[53]  D E Grobbee,et al.  External validation is necessary in prediction research: a clinical example. , 2003, Journal of clinical epidemiology.

[54]  A. Evans,et al.  Translating Clinical Research into Clinical Practice: Impact of Using Prediction Rules To Make Decisions , 2006, Annals of Internal Medicine.

[55]  A. Pesce,et al.  Continuous quality improvement in chronic disease: a computerized medical record enables description of a severity index to evaluate outcomes in end-stage renal disease. , 1992, American journal of kidney diseases : the official journal of the National Kidney Foundation.

[56]  Xueqing Yu,et al.  Risk score to predict mortality in continuous ambulatory peritoneal dialysis patients , 2014, European journal of clinical investigation.