PROBAST : A Tool to Assess the Risk of Bias and Applicability of PredictionModel Studies

Clinical prediction models combine multiple predictors to estimate risk for the presence of a particular condition (diagnostic models) or the occurrence of a certain event in the future (prognostic models). PROBAST (Prediction model Risk Of Bias ASsessment Tool), a tool for assessing the risk of bias (ROB) and applicability of diagnostic and prognostic prediction model studies, was developed by a steering group that considered existing ROB tools and reporting guidelines. The tool was informed by a Delphi procedure involving 38 experts and was refined through piloting. PROBAST is organized into the following 4 domains: participants, predictors, outcome, and analysis. These domains contain a total of 20 signaling questions to facilitate structured judgment of ROB, which was defined to occur when shortcomings in study design, conduct, or analysis lead to systematically distorted estimates of model predictive performance. PROBAST enables a focused and transparent approach to assessing the ROB and applicability of studies that develop, validate, or update prediction models for individualized predictions. Although PROBAST was designed for systematic reviews, it can be used more generally in critical appraisal of prediction model studies. Potential users include organizations supporting decision making, researchers and clinicians who are interested in evidence-based medicine or involved in guideline development, journal editors, and manuscript reviewers.

[1]  J. Higgins,et al.  Cochrane Handbook for Systematic Reviews of Interventions , 2010, International Coaching Psychology Review.

[2]  Iveta Simera,et al.  A proposed framework for developing quality assessment tools , 2017, Systematic Reviews.

[3]  L. Hooft,et al.  A guide to systematic review and meta-analysis of prediction model performance , 2017, British Medical Journal.

[4]  M. Hernán,et al.  ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions , 2016, British Medical Journal.

[5]  M. P. Koster,et al.  External validation of prognostic models to predict risk of gestational diabetes mellitus in one Dutch cohort: prospective multicentre cohort study , 2016, British Medical Journal.

[6]  G. Collins,et al.  Prediction models for cardiovascular disease risk in the general population: systematic review , 2016, British Medical Journal.

[7]  Rachel Churchill,et al.  ROBIS: A new tool to assess risk of bias in systematic reviews was developed , 2016, Journal of clinical epidemiology.

[8]  G. Collins,et al.  Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): The TRIPOD Statement , 2015, Annals of Internal Medicine.

[9]  Gary S Collins,et al.  Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration , 2015, Annals of Internal Medicine.

[10]  G. Collins,et al.  Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist , 2014, PLoS medicine.

[11]  Jennifer G. Robinson,et al.  2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk: A Report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines , 2014, Circulation.

[12]  Thoralf M Sundt,et al.  2014 AHA/ACC guideline for the management of patients with valvular heart disease: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines. , 2014, Journal of the American College of Cardiology.

[13]  Gary S Collins,et al.  A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods. , 2013, Journal of clinical epidemiology.

[14]  C. Bombardier,et al.  Assessing Bias in Studies of Prognostic Factors , 2013, Annals of Internal Medicine.

[15]  Richard D Riley,et al.  Prognosis research strategy (PROGRESS) 1: A framework for researching clinical outcomes , 2013, BMJ : British Medical Journal.

[16]  E. Steyerberg,et al.  Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research , 2013, PLoS medicine.

[17]  E. Steyerberg,et al.  Prognosis Research Strategy (PROGRESS) 2: Prognostic Factor Research , 2013, PLoS medicine.

[18]  Fiona Lecky,et al.  Predicting early death in patients with traumatic bleeding: development and validation of prognostic model , 2012, BMJ : British Medical Journal.

[19]  Silvia Rabar,et al.  Risk assessment of fragility fractures: summary of NICE guidance , 2012, BMJ : British Medical Journal.

[20]  E. Steyerberg,et al.  Reporting and Methods in Clinical Prediction Research: A Systematic Review , 2012, PLoS medicine.

[21]  Amanda H. Salanitro,et al.  Risk prediction models for hospital readmission: a systematic review. , 2011, JAMA.

[22]  Susan Mallett,et al.  Circulating MicroRNAs as a Novel Class of Diagnostic Biomarkers in Gastrointestinal Tumors Detection: A Meta-Analysis Based on 42 Articles , 2014, PloS one.

[23]  G. Collins,et al.  Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting , 2011, BMC medicine.

[24]  Lucas M. Bachmann,et al.  Clinical Value of Prognostic Instruments to Identify Patients with an Increased Risk for Osteoporotic Fractures: Systematic Review , 2011, PloS one.

[25]  Carmen Gomar,et al.  Prediction of Postoperative Pulmonary Complications in a Population-based Surgical Cohort , 2010, Anesthesiology.

[26]  Patrick Royston,et al.  Reporting methods in studies developing prognostic models in cancer: a review , 2010, BMC medicine.

[27]  D. Moher,et al.  Guidance for Developers of Health Research Reporting Guidelines , 2010, PLoS medicine.

[28]  P. Royston,et al.  Prognosis and prognostic research: application and impact of prognostic models in clinical practice , 2009, BMJ : British Medical Journal.

[29]  Yvonne Vergouwe,et al.  Prognosis and prognostic research: validating a prognostic model , 2009, BMJ : British Medical Journal.

[30]  Yvonne Vergouwe,et al.  Prognosis and prognostic research: Developing a prognostic model , 2009, BMJ : British Medical Journal.

[31]  Yvonne Vergouwe,et al.  Prognosis and prognostic research: what, why, and how? , 2009, BMJ : British Medical Journal.

[32]  Douglas G Altman,et al.  Prognostic Models: A Methodological Framework and Review of Models for Breast Cancer , 2009, Cancer investigation.

[33]  Shahrokh F. Shariat,et al.  Comparison of Nomograms With Other Methods for Predicting Outcomes in Prostate Cancer: A Critical Analysis of the Literature , 2008, Clinical Cancer Research.

[34]  A. Sheikh,et al.  Predicting cardiovascular risk in England and Wales: prospective derivation and validation of QRISK2 , 2008, BMJ : British Medical Journal.

[35]  Karel Moons,et al.  The Wells Rule Does Not Adequately Rule Out Deep Venous Thrombosis in Primary Care Patients , 2005, Annals of Internal Medicine.

[36]  Sunil J Rao,et al.  Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis , 2003 .

[37]  J André Knottnerus,et al.  Between iatrotropic stimulus and interiatric referral: the domain of primary care research. , 2002, Journal of clinical epidemiology.

[38]  M. Dennis,et al.  Systematic Review of Prognostic Models in Patients with Acute Stroke , 2001, Cerebrovascular Diseases.

[39]  G. Collins,et al.  PROBAST: A Tool to Assess Risk of Bias and Applicability of Prediction Model Studies: Explanation and Elaboration. , 2019, Annals of internal medicine.

[40]  Isabelle Boutron,et al.  A revised tool for assessing risk of bias in randomized trials , 2016 .

[41]  J. Knottnerus.  Diagnostic prediction rules: principles, requirements and pitfalls. , 1995, Primary care.