A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods.

BACKGROUND Chronic kidney disease (CKD) is a global health concern that is increasing mainly as the result of increasing incidences of diabetes and hypertension. Furthermore, if left untreated, individuals with CKD may progress to end-stage kidney failure. Identifying individuals with undiagnosed CKD or those who are at an increased risk of developing CKD or progressing to end-stage kidney disease (ESKD) is therefore an important challenge. We sought to systematically review and critically assess the conduct and reporting of methods used to develop risk prediction models for predicting the risk of having undiagnosed (prevalent) or future risk of developing (incident) CKD or end-stage kidney failure in adults. METHODS We conducted a systematic search of PubMed database to identify studies published up until September 2011 that describe the development of models combining two or more variables to predict the risk of prevalent or incident CKD or ESKD. We extracted key information that describes aspects of developing a prediction model, including the study design, data quality, sample size and number of events, outcome definition, risk predictor selection and coding, missing data, model-building strategies, and aspects of performance. RESULTS Eleven studies describing the development of 14 prediction models were included. Eight studies reported the development of 11 models to predict incident CKD or ESKD, whereas 3 studies developed models for prevalent CKD. A total of 97 candidate risk predictors were considered, and 43 different risk predictors featured in the 14 prediction models. A method, not recommended to select risk predictors for inclusion in the multivariate model, using statistical significance from univariate screening was carried out in six studies. Missing data were frequently poorly handled and reported with no mention of missing data in four studies; 4 studies explicitly excluded individuals with missing data, and only 2 studies used multiple imputation to replace missing values. CONCLUSION We found that prediction models for chronic kidney were often developed using inappropriate methods and were generally poorly reported. Using poor methods can affect the predictive ability of the models, whereas inadequate reporting hinders an objective evaluation of the potential usefulness of the model.

[1]  Ian Roberts,et al.  Systematic review of prognostic models in traumatic brain injury , 2006, BMC Medical Informatics Decis. Mak..

[2]  Patrick Royston,et al.  Reporting methods in studies developing prognostic models in cancer: a review , 2010, BMC medicine.

[3]  Daniel B. Mark,et al.  TUTORIAL IN BIOSTATISTICS MULTIVARIABLE PROGNOSTIC MODELS: ISSUES IN DEVELOPING MODELS, EVALUATING ASSUMPTIONS AND ADEQUACY, AND MEASURING AND REDUCING ERRORS , 1996 .

[4]  E. Steyerberg Clinical Prediction Models , 2008, Statistics for Biology and Health.

[5]  P. Roderick,et al.  A population-based study of the incidence and outcomes of diagnosed chronic kidney disease. , 2003, American journal of kidney diseases : the official journal of the National Kidney Foundation.

[6]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement , 2009, BMJ.

[7]  M. Mahdavi-Mazdeh,et al.  Establishing the Global Kidney Disease Prevention Network (KDPN): a position statement from the National Kidney Foundation. , 2011, American journal of kidney diseases : the official journal of the National Kidney Foundation.

[8]  Ewout W. Steyerberg,et al.  Medidas del rendimiento de modelos de predicción y marcadores pronósticos: evaluación de las predicciones y clasificaciones , 2011 .

[9]  Richard D. Riley,et al.  A systematic review of breast cancer incidence risk prediction models with meta-analysis of their performance , 2012, Breast Cancer Research and Treatment.

[10]  Douglas G Altman,et al.  [The Strengthening the Reporting of Observational Studies in Epidemiology [STROBE] statement: guidelines for reporting observational studies]. , 2007, Gaceta sanitaria.

[11]  D G Altman,et al.  REporting recommendations for tumour MARKer prognostic studies (REMARK) , 2005, British journal of cancer.

[12]  Carol Coupland,et al.  Predicting the risk of Chronic Kidney Disease in Men and Women in England and Wales: prospective derivation and external validation of the QKidney® Scores , 2010, BMC family practice.

[13]  Ewout W Steyerberg,et al.  A systematic review finds methodological improvements necessary for prognostic models in determining traumatic brain injury outcomes. , 2008, Journal of clinical epidemiology.

[14]  Alexander Gimson,et al.  Systematic review and validation of prognostic models in liver transplantation , 2005, Liver transplantation : official publication of the American Association for the Study of Liver Diseases and the International Liver Transplantation Society.

[15]  C. Gullion,et al.  Longitudinal follow-up and outcomes among a population with chronic kidney disease in a large managed care organization. , 2004, Archives of internal medicine.

[16]  Ewout W Steyerberg,et al.  Internal and external validation of predictive models: a simulation study of bias and precision in small samples. , 2003, Journal of clinical epidemiology.

[17]  Douglas G Altman,et al.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) Statement: guidelines for reporting observational studies. , 2014, International journal of surgery.

[18]  Heejung Bang,et al.  SCreening for Occult REnal Disease (SCORED): a simple prediction model for chronic kidney disease. , 2007, Archives of internal medicine.

[19]  Frank E. Harrell,et al.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis , 2001 .

[20]  D G Altman,et al.  What do we mean by validating a prognostic model? , 2000, Statistics in medicine.

[21]  K. Tsuchiya,et al.  A simple model for predicting incidence of chronic kidney disease in HIV-infected patients , 2011, Clinical and Experimental Nephrology.

[22]  R. Gansevoort,et al.  Development and validation of a general population renal risk score. , 2011, Clinical journal of the American Society of Nephrology : CJASN.

[23]  Simon J. Griffin,et al.  Risk Assessment Tools for Identifying Individuals at Risk of Developing Type 2 Diabetes , 2011, Epidemiologic reviews.

[24]  B. Hemmelgarn,et al.  Derivation and validation of a clinical index for prediction of rapid progression of kidney dysfunction. , 2006, QJM : monthly journal of the Association of Physicians.

[25]  N. Obuchowski,et al.  Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures , 2010, Epidemiology.

[26]  Charles E. McCulloch,et al.  CHRONIC KIDNEY DISEASE AND THE RISKS OF DEATH, CARDIOVASCULAR EVENTS, AND HOSPITALIZATION , 2004 .

[27]  J. Ioannidis,et al.  Strengthening the reporting of genetic risk prediction studies: the GRIPS statement , 2011, Genome Medicine.

[28]  Zhongxin Zhang,et al.  Risk scores for predicting outcomes in patients with type 2 diabetes and nephropathy: the RENAAL study. , 2006, Clinical journal of the American Society of Nephrology : CJASN.

[29]  P. Royston,et al.  Prognosis and prognostic research: application and impact of prognostic models in clinical practice , 2009, BMJ : British Medical Journal.

[30]  Hsiu-Ching Hsu,et al.  A prediction model for the risk of incident chronic kidney disease. , 2010, The American journal of medicine.

[31]  G. Remuzzi,et al.  The contribution of chronic kidney disease to the global burden of major noncommunicable diseases. , 2011, Kidney international.

[32]  G. Taylor,et al.  A Prediction Model for Chronic Kidney Disease Includes Periodontal Disease , 2009 .

[33]  Heejung Bang,et al.  A simple algorithm to predict incident kidney disease. , 2008, Archives of internal medicine.

[34]  J. Ioannidis,et al.  Strengthening the Reporting of Genetic Risk Prediction Studies: The GRIPS Statement , 2011, PLoS medicine.

[35]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA Statement , 2009, BMJ : British Medical Journal.

[36]  D. Moher,et al.  CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials , 2010, Journal of clinical epidemiology.

[37]  Douglas G Altman,et al.  Prognostic Models: A Methodological Framework and Review of Models for Breast Cancer , 2009, Cancer investigation.

[38]  S. Rattanasiri,et al.  A simplified clinical prediction score of chronic kidney disease: A cross-sectional-survey study , 2011, BMC nephrology.

[39]  M. Pencina,et al.  Performance Measures for Prediction Models and Markers: Evaluation of Predictions and Classifications , 2011 .

[40]  M. Woodward,et al.  Risk prediction models: II. External validation, model updating, and impact assessment , 2012, Heart.

[41]  P. Stevens,et al.  Unreferred chronic kidney disease: a longitudinal study. , 2004, American journal of kidney diseases : the official journal of the National Kidney Foundation.

[42]  G. Collins,et al.  Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting , 2011, BMC medicine.

[43]  Ewout W Steyerberg,et al.  Some prognostic models for traumatic brain injury were not valid. , 2006, Journal of clinical epidemiology.

[44]  R. D'Agostino,et al.  Presentation of multivariate data for clinical use: The Framingham Study risk score functions , 2004, Statistics in medicine.

[45]  J. Bader,et al.  Root caries risk indicators: a systematic review of risk models. , 2010, Community dentistry and oral epidemiology.

[46]  D. G. Altman,et al.  Review of survival analyses published in cancer journals. , 1995, British Journal of Cancer.

[47]  N. Tangri,et al.  A predictive model for progression of chronic kidney disease to kidney failure. , 2011, JAMA.

[48]  S W Lagakos,et al.  Effects of mismodelling and mismeasuring explanatory variables on tests of their association with a response variable. , 1988, Statistics in medicine.

[49]  David Moher,et al.  Towards complete and accurate reporting of studies of diagnostic accuracy: the STARD initiative. Standards for Reporting of Diagnostic Accuracy. , 2003, Clinical chemistry.

[50]  W. Sauerbrei,et al.  Reporting recommendations for tumor marker prognostic studies (REMARK). , 2005, Journal of the National Cancer Institute.

[51]  Patrick Royston,et al.  Reporting performance of prognostic models in cancer: a review , 2010, BMC medicine.

[52]  Trisha Greenhalgh,et al.  Risk models and scores for type 2 diabetes: systematic review , 2011, BMJ : British Medical Journal.