论文信息 - Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist

Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist

Carl Moons and colleagues provide a checklist and background explanation for critically appraising and extracting data from systematic reviews of prognostic and diagnostic prediction modelling studies. Please see later in the article for the Editors' Summary

[1] Jennifer G. Robinson,et al. 2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk: A Report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines , 2014, Circulation.

[2] Jennifer G. Robinson,et al. 2013 ACC/AHA Guideline on the Assessment of Cardiovascular Risk: A Report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines , 2014, Circulation.

[3] M. Drazner,et al. 2013 ACCF/AHA guideline for the management of heart failure: a report of the American College of Cardiology Foundation/American Heart Association Task Force on Practice Guidelines. , 2013, Journal of the American College of Cardiology.

[4] P. Royston,et al. External validation of a Cox prognostic model: principles and methods , 2013, BMC Medical Research Methodology.

[5] Gary S Collins,et al. A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods. , 2013, Journal of clinical epidemiology.

[6] C. Bombardier,et al. Assessing Bias in Studies of Prognostic Factors , 2013, Annals of Internal Medicine.

[7] E. Steyerberg,et al. Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research , 2013, PLoS medicine.

[8] G. Gamble,et al. Discrepancies in predicted fracture risk in elderly people , 2013, BMJ.

[9] Bas van Zaane,et al. Comparison of approaches to estimate confidence intervals of post-test probabilities of diagnostic test results in a nested case-control study , 2012, BMC Medical Research Methodology.

[10] K. Moons,et al. A simple adaptation method improved the interpretability of prediction models for composite end points. , 2012, Journal of clinical epidemiology.

[11] Silvia Rabar,et al. Risk assessment of fragility fractures: summary of NICE guidance , 2012, BMJ : British Medical Journal.

[12] Gary S Collins,et al. Predicting the 10 year risk of cardiovascular disease in the United Kingdom: independent and external validation of an updated version of QRISK2 , 2012, BMJ : British Medical Journal.

[13] Gary S Collins,et al. Comparing risk prediction models , 2012, BMJ : British Medical Journal.

[14] John P A Ioannidis,et al. Comparisons of established risk prediction models for cardiovascular disease: systematic review , 2012, BMJ : British Medical Journal.

[15] E. Steyerberg,et al. Reporting and Methods in Clinical Prediction Research: A Systematic Review , 2012, PLoS medicine.

[16] Patrik Magnusson,et al. American Journal of Epidemiology Practice of Epidemiology Risk Prediction Measures for Case-cohort and Nested Case-control Designs: an Application to Cardiovascular Disease , 2022 .

[17] Rolf H H Groenwold,et al. Performance of the original EuroSCORE. , 2012, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[18] M. Woodward,et al. Risk prediction models: II. External validation, model updating, and impact assessment , 2012, Heart.

[19] M. Woodward,et al. Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker , 2012, Heart.

[20] A. Vickers,et al. Against quantiles: categorization of continuous variables in epidemiologic research, and its discontents , 2012, BMC Medical Research Methodology.

[21] M. Leeflang,et al. Search Filters for Finding Prognostic and Diagnostic Prediction Studies in Medline to Enhance Systematic Reviews , 2012, PloS one.

[22] E. Mohammadi,et al. Barriers and facilitators related to the implementation of a physiological track and trigger system: A systematic review of the qualitative evidence , 2017, International journal for quality in health care : journal of the International Society for Quality in Health Care.

[23] K G M Moons,et al. Prediction models for the risk of cardiovascular disease in patients with type 2 diabetes: a systematic review , 2011, Heart.

[24] S. Peters,et al. Improvements in risk stratification for the occurrence of cardiovascular disease by imaging subclinical atherosclerosis: a systematic review , 2011, Heart.

[25] Ameen Abu-Hanna,et al. Effect of changes over time in the performance of a customized SAPS-II model on the quality of care assessment , 2013 .

[26] Susan Mallett,et al. Circulating MicroRNAs as a Novel Class of Diagnostic Biomarkers in Gastrointestinal Tumors Detection: A Meta-Analysis Based on 42 Articles , 2014, PloS one.

[27] J. Sterne,et al. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials , 2011, BMJ : British Medical Journal.

[28] G. Collins,et al. Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting , 2011, BMC medicine.

[29] Thomas Agoritsas,et al. Performance of logistic regression modeling: beyond the number of events per variable, the role of data structure. , 2011, Journal of clinical epidemiology.

[30] Tom Fahey,et al. Optimized retrieval of primary care clinical prediction rules from MEDLINE to establish a Web-based register. , 2011, Journal of clinical epidemiology.

[31] E. Steyerberg,et al. [Regression modeling strategies]. , 2011, Revista espanola de cardiologia.

[32] Margaret S Pepe,et al. Problems with risk reclassification methods for evaluating prediction models. , 2011, American journal of epidemiology.

[33] Gert Kwakkel,et al. Early Prediction of Outcome of Activities of Daily Living After Stroke: A Systematic Review , 2011, Stroke.

[34] J. Gulcher,et al. Strengthening the reporting of genetic risk prediction studies (GRIPS): explanation and elaboration , 2011, European Journal of Human Genetics.

[35] Patrick Royston,et al. Multiple imputation using chained equations: Issues and guidance for practice , 2011, Statistics in medicine.

[36] Ewout W Steyerberg,et al. Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers , 2011, Statistics in medicine.

[37] A. Mackinnon,et al. The use and reporting of multiple imputation in medical research – a review , 2010, Journal of internal medicine.

[38] Andrew J Vickers,et al. Everything you always wanted to know about evaluating prediction models (but were too afraid to ask). , 2010, Urology.

[39] Yvonne Vergouwe,et al. External validity of risk models: Use of benchmark values to disentangle a case-mix effect from incorrect coefficients. , 2010, American journal of epidemiology.

[40] Victor M Montori,et al. Accuracy and quality of clinical decision rules for syncope in the emergency department: a systematic review and meta-analysis. , 2010, Annals of emergency medicine.

[41] A. Nierich,et al. Prediction Models for Prolonged Intensive Care Unit Stay After Cardiac Surgery: Systematic Review and Validation Study , 2010, Circulation.

[42] Qingxia Chen,et al. Missing covariate data in medical research: to impute is better than to ignore. , 2010, Journal of clinical epidemiology.

[43] Patrick Royston,et al. Reporting methods in studies developing prognostic models in cancer: a review , 2010, BMC medicine.

[44] Patrick Royston,et al. Reporting performance of prognostic models in cancer: a review , 2010, BMC medicine.

[45] D. Moher,et al. CONSORT 2010 Explanation and Elaboration: updated guidelines for reporting parallel group randomised trials , 2011, BMJ : British Medical Journal.

[46] Yvonne Vergouwe,et al. Development and validation of a prediction model with missing predictor data: a practical approach. , 2010, Journal of clinical epidemiology.

[47] Douglas G Altman,et al. Comparison of techniques for handling missing covariate data within prognostic modelling studies: a simulation study , 2010, BMC medical research methodology.

[48] N. Obuchowski,et al. Assessing the Performance of Prediction Models: A Framework for Traditional and Novel Measures , 2010, Epidemiology.

[49] W. Sauerbrei,et al. Reporting of prognostic studies of tumour markers: a review of published articles in relation to REMARK guidelines , 2009, British Journal of Cancer.

[50] D. Moher,et al. CONSORT 2010 explanation and elaboration: Updated guidelines for reporting parallel group randomised trialsqqq , 2018 .

[51] J. Ioannidis,et al. Assessment of claims of improved prediction beyond the Framingham risk score. , 2009, JAMA.

[52] Patrick M M Bossuyt,et al. Prediction models in reproductive medicine: a critical appraisal. , 2009, Human reproduction update.

[53] J. Ioannidis,et al. The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions : Explanation and Elaboration , 2009 .

[54] Johannes B. Reitsma,et al. A review of solutions for diagnostic accuracy studies with an imperfect or missing reference standard. , 2009, Journal of clinical epidemiology.

[55] J. Ioannidis,et al. The PRISMA Statement for Reporting Systematic Reviews and Meta-Analyses of Studies That Evaluate Health Care Interventions: Explanation and Elaboration , 2009, Annals of Internal Medicine [serial online].

[56] Kathy Boutis,et al. Should a Head-Injured Child Receive a Head CT Scan? A Systematic Review of Clinical Prediction Rules , 2009, Pediatrics.

[57] M. Kenward,et al. Multiple imputation for missing data in epidemiological and clinical research: potential and pitfalls , 2009, BMJ : British Medical Journal.

[58] C.J.H. Mann,et al. Clinical Prediction Models: A Practical Approach to Development, Validation and Updating , 2009 .

[59] P. Royston,et al. Prognosis and prognostic research: application and impact of prognostic models in clinical practice , 2009, BMJ : British Medical Journal.

[60] Yvonne Vergouwe,et al. Prognosis and prognostic research: validating a prognostic model , 2009, BMJ : British Medical Journal.

[61] Qingxia Chen,et al. Dealing with missing predictor values when applying clinical prediction models. , 2009, Clinical chemistry.

[62] Yvonne Vergouwe,et al. Prognosis and prognostic research: Developing a prognostic model , 2009, BMJ : British Medical Journal.

[63] Yvonne Vergouwe,et al. Prognosis and prognostic research: what, why, and how? , 2009, BMJ : British Medical Journal.

[64] S. George,et al. Clinical Prediction Rules for Physical Therapy Interventions: A Systematic Review , 2009, Physical Therapy.

[65] J. Cnossen,et al. The Accuracy of Risk Scores in Predicting Ovarian Malignancy: A Systematic Review , 2009, Obstetrics and gynecology.

[66] Douglas G Altman,et al. Prognostic Models: A Methodological Framework and Review of Models for Breast Cancer , 2009, Cancer investigation.

[67] L. Irwig,et al. Assessing new biomarkers and predictive models for use in clinical practice: a clinician's guide. , 2008, Archives of internal medicine.

[68] Y. Vergouwe,et al. Validation, updating and impact of clinical prediction rules: a review. , 2008, Journal of clinical epidemiology.

[69] E. Steyerberg. Clinical Prediction Models , 2008, Statistics for Biology and Health.

[70] Patrick Royston,et al. How should variable selection be performed with multiply imputed data? , 2008, Statistics in medicine.

[71] Yvonne Vergouwe,et al. Bmc Medical Research Methodology Open Access Advantages of the Nested Case-control Design in Diagnostic Research , 2022 .

[72] E. Hess,et al. Diagnostic accuracy of clinical prediction rules to exclude acute coronary syndrome in the emergency department setting: a systematic review. , 2008, CJEM.

[73] Shahrokh F. Shariat,et al. Inventory of prostate cancer predictive tools , 2008, Current opinion in urology.

[74] M. Leeflang,et al. Bias in sensitivity and specificity caused by data-driven selection of optimal cutoff values: mechanisms, magnitude, and solutions. , 2008, Clinical chemistry.

[75] Diederick E. Grobbee,et al. Clinical Epidemiology: Principles, Methods, and Applications for Clinical Research , 2008 .

[76] Nancy R Cook,et al. Comments on ‘Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond’ by M. J. Pencina et al., Statistics in Medicine (DOI: 10.1002/sim.2929) , 2008, Statistics in medicine.

[77] M. Pencina,et al. Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond , 2008, Statistics in medicine.

[78] Y Vergouwe,et al. Updating methods improved the performance of a clinical prediction model in new patients. , 2008, Journal of clinical epidemiology.

[79] N. Cook. Statistical evaluation of prognostic versus diagnostic models: beyond the ROC curve. , 2008, Clinical chemistry.

[80] P. Bossuyt,et al. Evaluation of diagnostic tests when there is no gold standard. A review of methods. , 2007, Health technology assessment.

[81] G. Lyman,et al. Breast Cancer—Translational Therapeutic Strategies , 2007 .

[82] S. Pocock,et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. , 2007, Preventive medicine.

[83] Richard D Riley,et al. Prognosis research: toward evidence-based results and a Cochrane methods group. , 2007, Journal of clinical epidemiology.

[84] N F de Keizer,et al. External validation of prognostic models for critically ill patients required substantial sample sizes. , 2007, Journal of clinical epidemiology.

[85] Gordon H Guyatt,et al. Problems with use of composite end points in cardiovascular trials: systematic review of randomised controlled trials , 2007, BMJ : British Medical Journal.

[86] Charles E McCulloch,et al. Relaxing the rule of ten events per variable in logistic and Cox regression. , 2007, American journal of epidemiology.

[87] J. Ioannidis,et al. Quality of reporting of cancer prognostic marker studies: association with reported prognostic effect. , 2007, Journal of the National Cancer Institute.

[88] H. Hemingway. Prognosis research: why is Dr. Lydgate still waiting? , 2006, Journal of clinical epidemiology.

[89] Ian Roberts,et al. Systematic review of prognostic models in traumatic brain injury , 2006, BMC Medical Informatics Decis. Mak..

[90] E. Elkin,et al. Decision Curve Analysis: A Novel Method for Evaluating Prediction Models , 2006, Medical decision making : an international journal of the Society for Medical Decision Making.

[91] M. Gorelick,et al. Bias arising from missing data in predictive models. , 2006, Journal of clinical epidemiology.

[92] Theo Stijnen,et al. Using the outcome for imputation of missing predictor values was preferred. , 2006, Journal of clinical epidemiology.

[93] T. Stijnen,et al. Review: a gentle introduction to imputation of missing values. , 2006, Journal of clinical epidemiology.

[94] Patrick Royston,et al. The cost of dichotomising continuous variables , 2006, BMJ : British Medical Journal.

[95] Jill Hayden,et al. Evaluation of the Quality of Prognosis Studies in Systematic Reviews , 2006, Annals of Internal Medicine.

[96] Johannes B Reitsma,et al. Evidence of bias and variation in diagnostic accuracy studies , 2006, Canadian Medical Association Journal.

[97] A. Evans,et al. Translating Clinical Research into Clinical Practice: Impact of Using Prediction Rules To Make Decisions , 2006, Annals of Internal Medicine.

[98] Ewout W Steyerberg,et al. Some prognostic models for traumatic brain injury were not valid. , 2006, Journal of clinical epidemiology.

[99] Douglas G Altman,et al. Dichotomizing continuous predictors in multiple regression: a bad idea , 2006, Statistics in medicine.

[100] A. Harris,et al. REporting recommendations for tumour MARKer prognostic studies (REMARK) , 2005, British Journal of Cancer.

[101] Johannes B Reitsma,et al. Case-control and two-gate designs in diagnostic accuracy studies. , 2005, Clinical chemistry.

[102] Karel G M Moons,et al. Ruling out deep venous thrombosis in primary care , 2005, Thrombosis and Haemostasis.

[103] Yvonne Vergouwe,et al. Substantial effective sample sizes were required for external validation studies of predictive logistic regression models. , 2005, Journal of clinical epidemiology.

[104] U. Finsterer,et al. Development and validation of a mathematical algorithm for quantifying preoperative blood volume by means of the decrease in hematocrit resulting from acute normovolemic hemodilution , 2005, Transfusion.

[105] A. Hoes,et al. Limited value of patient history and physical examination in diagnosing deep vein thrombosis in primary care. , 2004, Family practice.

[106] I. Ellis,et al. The Nottingham prognostic index in primary breast cancer , 2005, Breast Cancer Research and Treatment.

[107] L. Tamariz,et al. Usefulness of clinical prediction rules for the diagnosis of venous thromboembolism: a systematic review. , 2004, The American journal of medicine.

[108] Carl van Walraven,et al. Time-dependent bias was common in survival analyses published in leading clinical journals. , 2004, Journal of clinical epidemiology.

[109] Gerben ter Riet,et al. Predicting bacterial cause in infectious conjunctivitis: cohort study on informativeness of combinations of signs and symptoms , 2004, BMJ : British Medical Journal.

[110] D G Altman,et al. Missing covariate data within cancer prognostic studies: a review of current reporting and proposed guidelines , 2004, British Journal of Cancer.

[111] P. Bossuyt,et al. Sources of Variation and Bias in Studies of Diagnostic Accuracy , 2004, Annals of Internal Medicine.

[112] R. Glynn,et al. Methods to evaluate risks for composite end points and their individual components. , 2004, Journal of clinical epidemiology.

[113] Douglas G. Altman,et al. Methodological challenges in the evaluation of prognostic factors in breast cancer , 2004, Breast Cancer Research and Treatment.

[114] P. Bossuyt,et al. BMC Medical Research Methodology , 2002 .

[115] Karel G M Moons,et al. Diagnostic research on routine care data: prospects and problems. , 2003, Journal of clinical epidemiology.

[116] J. Habbema,et al. External validity of a prediction rule for residual mass histology in testicular cancer: an evaluation for good prognosis patients , 2003, British Journal of Cancer.

[117] Jaakko Tuomilehto,et al. The diabetes risk score: a practical tool to predict type 2 diabetes risk. , 2003, Diabetes care.

[118] David Moher,et al. The STARD Statement for Reporting Studies of Diagnostic Accuracy: Explanation and Elaboration , 2003, Annals of Internal Medicine [serial online].

[119] R. Brian Haynes,et al. Developing Optimal Search Strategies for Detecting Sound Clinical Prediction Studies in MEDLINE , 2003, AMIA.

[120] Diederick E Grobbee,et al. When should we remain blind and when should our eyes remain open in diagnostic studies? , 2002, Journal of clinical epidemiology.

[121] J. Knottnerus. Challenges in dia-prognostic research , 2002, Journal of epidemiology and community health.

[122] D. Grobbee,et al. Diagnostic studies as multivariable, prediction research , 2002, Journal of epidemiology and community health.

[123] M. Dennis,et al. Systematic Review of Prognostic Models in Patients with Acute Stroke , 2001, Cerebrovascular Diseases.

[124] Douglas G Altman,et al. Systematic reviews of evaluations of prognostic variables , 2001, BMJ : British Medical Journal.

[125] D. Moher,et al. The Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration , 2001, Annals of Internal Medicine.

[126] J. Habbema,et al. Prognostic Modeling with Logistic Regression Analysis , 2001, Medical decision making : an international journal of the Society for Medical Decision Making.

[127] Frank E. Harrell,et al. Regression Modeling Strategies: With Applications to Linear Models, Logistic Regression, and Survival Analysis , 2001 .

[128] B. J. Ingui,et al. Searching for clinical prediction rules in MEDLINE. , 2001, Journal of the American Medical Informatics Association : JAMIA.

[129] Hans C. van Houwelingen,et al. Validation, calibration, revision and combination of prognostic survival models , 2000 .

[130] G H Guyatt,et al. Users' guides to the medical literature: XXII: how to use articles about clinical decision rules. Evidence-Based Medicine Working Group. , 2000, JAMA.

[131] J. Habbema,et al. Prognostic modelling with logistic regression analysis: a comparison of selection and estimation methods in small data sets. , 2000, Statistics in medicine.

[132] D G Altman,et al. What do we mean by validating a prognostic model? , 2000, Statistics in medicine.

[133] D G Altman,et al. Modeling the effects of continuous risk factors. , 2000, Journal of clinical epidemiology.

[134] H C van Houwelingen,et al. Validation, calibration, revision and combination of prognostic survival models. , 2000, Statistics in medicine.

[135] P. Bossuyt,et al. Empirical evidence of design-related bias in studies of diagnostic tests. , 1999, JAMA.

[136] S. Lemeshow,et al. European system for cardiac operative risk evaluation (EuroSCORE). , 1999, European journal of cardio-thoracic surgery : official journal of the European Association for Cardio-thoracic Surgery.

[137] I. Stiell,et al. Methodologic standards for the development of clinical decision rules in emergency medicine. , 1999, Annals of emergency medicine.

[138] K. Covinsky,et al. Assessing the Generalizability of Prognostic Information , 1999, Annals of Internal Medicine.

[139] S. Nashef,et al. Eurp-pean system for cardiac operative risk evaluation (Euro SCORE) , 1999 .

[140] D. Levy,et al. Prediction of coronary heart disease using risk factor categories. , 1998, Circulation.

[141] J. Hirsh,et al. A simple clinical model for the diagnosis of deep‐vein thrombosis combined with impedance plethysmography: potential for an improvement in the diagnostic process , 1998, Journal of internal medicine.

[142] A. Laupacis,et al. Clinical prediction rules. A review and suggested modifications of methodological standards. , 1997, JAMA.

[143] J. Concato,et al. A simulation study of the number of events per variable in logistic regression analysis. , 1996, Journal of clinical epidemiology.

[144] G W Sun,et al. Inappropriate use of bivariable analysis to screen risk factors for use in multivariable analysis. , 1996, Journal of clinical epidemiology.

[145] F. Harrell,et al. Prognostic/Clinical Prediction Models: Multivariable Prognostic Models: Issues in Developing Models, Evaluating Assumptions and Adequacy, and Measuring and Reducing Errors , 2005 .

[146] J. Concato,et al. Importance of events per independent variable in proportional hazards regression analysis. II. Accuracy and precision of regression estimates. , 1995, Journal of clinical epidemiology.

[147] P Peduzzi,et al. Importance of events per independent variable in proportional hazards analysis. I. Background, goals, and general strategy. , 1995, Journal of clinical epidemiology.

[148] E. Safranek,et al. Decision rules for the use of radiography in acute ankle injuries , 1994 .