A guide to systematic review and meta-analysis of prognostic factor studies

Prognostic factors are associated with the risk of future health outcomes in individuals with a particular health condition or some clinical start point (eg, a particular diagnosis). Research to identify genuine prognostic factors is important because these factors can help improve risk stratification, treatment, and lifestyle decisions, and the design of randomised trials. Although thousands of prognostic factor studies are published each year, often they are of variable quality and the findings are inconsistent. Systematic reviews and meta-analyses are therefore needed that summarise the evidence about the prognostic value of particular factors. In this article, the key steps involved in this review process are described.

[1]  Richard D Riley,et al.  Prognosis research strategy (PROGRESS) 1: A framework for researching clinical outcomes , 2013, BMJ : British Medical Journal.

[2]  E. Steyerberg,et al.  Prognosis Research Strategy (PROGRESS) 3: Prognostic Model Research , 2013, PLoS medicine.

[3]  A. Sutton,et al.  Comparison of two methods to detect publication bias in meta-analysis. , 2006, JAMA.

[4]  L. Hedges,et al.  Introduction to Meta‐Analysis , 2009, International Coaching Psychology Review.

[5]  B. J. Ingui,et al.  Searching for clinical prediction rules in MEDLINE. , 2001, Journal of the American Medical Informatics Association : JAMIA.

[6]  David R. Jones,et al.  A systematic review and evaluation of the use of tumour markers in paediatric oncology: Ewing's sarcoma and neuroblastoma. , 2003, Health technology assessment.

[7]  Richard D Riley,et al.  Interpretation of random effects meta-analyses , 2011, BMJ : British Medical Journal.

[8]  J. Ioannidis,et al.  Recommendations for examining and interpreting funnel plot asymmetry in meta-analyses of randomised controlled trials , 2011, BMJ : British Medical Journal.

[9]  John P A Ioannidis,et al.  Selective reporting biases in cancer prognostic factor studies. , 2005, Journal of the National Cancer Institute.

[10]  R. Haynes,et al.  Medline : analytical survey scientifically strong studies of diagnosis from Optimal search strategies for retrieving , 2004 .

[11]  S Greenland,et al.  Meta‐analysis of Epidemiologic Dose‐Response Data , 1993, Epidemiology.

[12]  Patrick Royston,et al.  A new measure of prognostic separation in survival data , 2004, Statistics in medicine.

[13]  G. Collins,et al.  External validation of multivariable prediction models: a systematic review of methodological conduct and reporting , 2014, BMC Medical Research Methodology.

[14]  E. Steyerberg,et al.  Prognosis Research Strategy (PROGRESS) 2: Prognostic Factor Research , 2013, PLoS medicine.

[15]  Douglas G Altman,et al.  Systematic reviews of evaluations of prognostic variables , 2001, BMJ : British Medical Journal.

[16]  Patrick Bossuyt,et al.  Systematic Reviews of Diagnostic Test Accuracy , 2008, Annals of Internal Medicine.

[17]  Larry V. Hedges,et al.  Converting Among Effect Sizes , 2009, Introduction to Meta‐Analysis.

[18]  D G Altman,et al.  Prognostic markers in cancer: the evolution of evidence from single studies to meta-analysis, and beyond , 2009, British Journal of Cancer.

[19]  Richard D Riley,et al.  Individual participant data meta-analysis of prognostic factor studies: state of the art? , 2012, BMC Medical Research Methodology.

[20]  Richard D Riley,et al.  Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples , 2017, British Medical Journal.

[21]  R. Brian Haynes,et al.  Developing Optimal Search Strategies for Detecting Sound Clinical Prediction Studies in MEDLINE , 2003, AMIA.

[22]  Gary S Collins,et al.  A systematic review finds prediction models for chronic kidney disease were poorly reported and often developed using inappropriate methods. , 2013, Journal of clinical epidemiology.

[23]  T. Perneger.  Estimating the relative hazard by the ratio of logarithms of event-free proportions. , 2008, Contemporary clinical trials.

[24]  Jian Qing Shi,et al.  Meta‐analysis for trend estimation , 2004, Statistics in medicine.

[25]  I. Olkin,et al.  Meta-analysis of observational studies in epidemiology - A proposal for reporting , 2000 .

[26]  G. Collins,et al.  Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies: The CHARMS Checklist , 2014, PLoS medicine.

[27]  C. Bombardier,et al.  Assessing Bias in Studies of Prognostic Factors , 2013, Annals of Internal Medicine.

[28]  Douglas G. Altman,et al.  Reporting Recommendations for Tumor Marker Prognostic Studies (REMARK): Explanation and Elaboration , 2012, PLoS medicine.

[29]  Kunihiko Takahashi,et al.  Meta‐analysis of prognostic studies for a biomarker with a study‐specific cutoff value , 2016, Research synthesis methods.

[30]  A. Harris,et al.  REporting recommendations for tumour MARKer prognostic studies (REMARK) , 2005, British Journal of Cancer.

[31]  D. Firth.  Bias reduction of maximum likelihood estimates , 1993 .

[32]  Richard D. Riley,et al.  Random effects meta‐analysis: Coverage performance of 95% confidence and prediction intervals following REML estimation , 2016, Statistics in medicine.

[33]  M. Leeflang,et al.  Search Filters for Finding Prognostic and Diagnostic Prediction Studies in Medline to Enhance Systematic Reviews , 2012, PloS one.

[34]  H Tunstall-Pedoe,et al.  Systematically missing confounders in individual participant data meta-analysis of observational cohort studies , 2009, Statistics in medicine.

[35]  D G Altman,et al.  REporting recommendations for tumour MARKer prognostic studies (REMARK) , 2005, British journal of cancer.

[36]  A. Harris,et al.  Systematic Review of Multiple Studies of Prognosis: The Feasibility of Obtaining Individual Patient Data , 2007 .

[37]  Gary L Myers,et al.  Markers of inflammation and cardiovascular disease: application to clinical and public health practice: A statement for healthcare professionals from the Centers for Disease Control and Prevention and the American Heart Association. , 2003, Circulation.

[38]  M. Hernán,et al.  ROBINS-I: a tool for assessing risk of bias in non-randomised studies of interventions , 2016, British Medical Journal.

[39]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. , 2010, International journal of surgery.

[40]  M. Sydes,et al.  Practical methods for incorporating summary time-to-event data into meta-analysis , 2007, Trials.

[41]  J. Sterne,et al.  How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. , 2003, Health technology assessment.

[42]  Douglas G. Altman,et al.  Improving the Transparency of Prognosis Research: The Role of Reporting, Data Sharing, Registration, and Protocols , 2014, PLoS medicine.

[43]  G. Collins,et al.  Developing risk prediction models for type 2 diabetes: a systematic review of methodology and reporting , 2011, BMC medicine.

[44]  Yemisi Takwoingi,et al.  Empirical Evidence of the Importance of Comparative Studies of Diagnostic Test Accuracy , 2013, Annals of Internal Medicine.

[45]  Mark Woodward,et al.  Assessing Risk Prediction Models Using Individual Participant Data From Multiple Studies , 2013, American journal of epidemiology.

[46]  Richard D Riley,et al.  Prognosis research strategy (PROGRESS) 4: Stratified medicine research , 2013, BMJ : British Medical Journal.

[47]  S. Greenland,et al.  Methods for trend estimation from summarized dose-response data, with applications to meta-analysis. , 1992, American journal of epidemiology.

[48]  K. A. McKibbon,et al.  Optimal search strategies for retrieving scientifically strong studies of treatment from Medline: analytical survey , 2005, BMJ : British Medical Journal.

[49]  P. Tugwell,et al.  The Newcastle-Ottawa Scale (NOS) for Assessing the Quality of Nonrandomised Studies in Meta-Analyses , 2014 .

[50]  Patrick Royston,et al.  Reporting methods in studies developing prognostic models in cancer: a review , 2010, BMC medicine.

[51]  P C Lambert,et al.  Reporting of prognostic markers: current problems and development of guidelines for evidence-based practice in the future , 2003, British Journal of Cancer.

[52]  Gerta Rücker,et al.  Undue reliance on I2 in assessing heterogeneity may mislead , 2008, BMC medical research methodology.

[53]  David J Spiegelhalter,et al.  A re-evaluation of random-effects meta-analysis , 2009, Journal of the Royal Statistical Society. Series A,.

[54]  Richard D Riley,et al.  A framework for meta-analysis of prediction model studies with binary and time-to-event outcomes , 2018, Statistical methods in medical research.

[55]  C S Berkey,et al.  A random-effects regression model for meta-analysis. , 1995, Statistics in medicine.

[56]  L. Hooft,et al.  A guide to systematic review and meta-analysis of prediction model performance , 2017, British Medical Journal.

[57]  Patrick Royston,et al.  Explained Variation for Survival Models , 2006 .

[58]  M. Pencina,et al.  Evaluating the added predictive ability of a new marker: From area under the ROC curve to reclassification and beyond , 2008, Statistics in medicine.

[59]  Nicola Orsini,et al.  Meta-analysis for linear and nonlinear dose-response relations: examples, an evaluation of approximations, and software. , 2012, American journal of epidemiology.

[60]  E. Steyerberg,et al.  Reporting and Methods in Clinical Prediction Research: A Systematic Review , 2012, PLoS medicine.

[61]  Richard D Riley,et al.  Evidence-Based Assessment and Application of Prognostic Markers: The Long Way from Single Studies to Meta-Analysis , 2006 .

[62]  R. Riley,et al.  Detecting small‐study effects and funnel plot asymmetry in meta‐analysis of survival data: A comparison of new and existing tests , 2017, Research synthesis methods.

[63]  M. Parmar,et al.  Extracting summary statistics to perform meta-analyses of the published literature for survival endpoints. , 1998, Statistics in medicine.

[64]  Gary S Collins,et al.  Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD): Explanation and Elaboration , 2015, Annals of Internal Medicine.

[65]  D G Altman,et al.  Missing covariate data within cancer prognostic studies: a review of current reporting and proposed guidelines , 2004, British Journal of Cancer.

[66]  Bette Jean Ingui,et al.  Research Paper: Searching for Clinical Prediction Rules in MEDLINE , 2001, J. Am. Medical Informatics Assoc..

[67]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement , 2009, BMJ.

[68]  Keith R. Abrams,et al.  Evaluating the Quality of Research into a Single Prognostic Biomarker: A Systematic Review and Meta-analysis of 83 Studies of C-Reactive Protein in Stable Coronary Artery Disease , 2010, PLoS medicine.

[69]  Karla Hemming,et al.  Multivariate meta‐analysis of prognostic factor studies with multiple cut‐points and/or methods of measurement , 2015, Statistics in medicine.

[70]  Richard D Riley,et al.  Primer: an evidence-based approach to prognostic markers , 2005, Nature Clinical Practice Oncology.

[71]  G. Guyatt,et al.  Use of GRADE for assessment of evidence about prognosis: rating confidence in estimates of event rates in broad categories of patients , 2015, BMJ : British Medical Journal.

[72]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[73]  J. Auget,et al.  Advances in statistical methods for the health sciences : applications to cancer and AIDS studies, genome sequence analysis, and survival analysis , 2007 .

[74]  Patrick Royston,et al.  Reporting performance of prognostic models in cancer: a review , 2010, BMC medicine.

[75]  R D Riley,et al.  Random‐effects meta‐analysis of the clinical utility of tests and prediction models , 2018, Statistics in medicine.

[76]  John P A Ioannidis,et al.  Almost all articles on cancer prognostic markers report statistically significant results. , 2007, European journal of cancer.

[77]  Douglas G Altman,et al.  REporting recommendations for tumour MARKer prognostic studies (REMARK). , 2005, European journal of cancer.

[78]  Douglas G Altman,et al.  Reporting recommendations for tumor marker prognostic studies (REMARK): explanation and elaboration , 2012, BMC Medicine.

[79]  R. Riley,et al.  Meta-analysis of individual participant data: rationale, conduct, and reporting , 2010, BMJ : British Medical Journal.

[80]  Hendriek C Boshuizen,et al.  Combining risk estimates from observational studies with different exposure cutpoints: a meta-analysis on body mass index and diabetes type 2. , 2006, American journal of epidemiology.

[81]  J. Ioannidis,et al.  Quality of reporting of cancer prognostic marker studies: association with reported prognostic effect. , 2007, Journal of the National Cancer Institute.

[82]  L. Hooft,et al.  Implementing systematic reviews of prognosis studies in Cochrane. , 2018, The Cochrane database of systematic reviews.

[83]  Cynthia D Mulrow,et al.  Random-Effects Meta-analysis of Inconsistent Effects: A Time for Change , 2014, Annals of Internal Medicine.

[84]  J. Hartung,et al.  A refined method for the meta‐analysis of controlled clinical trials with binary outcome , 2001, Statistics in medicine.

[85]  Ewout W Steyerberg,et al.  Net benefit approaches to the evaluation of prediction models, molecular markers, and diagnostic tests , 2016, British Medical Journal.

[86]  Julian P T Higgins,et al.  An empirical comparison of heterogeneity variance estimators in 12 894 meta‐analyses , 2015, Research synthesis methods.

[87]  T Pérez,et al.  Extraction of unadjusted estimates of prognostic association for meta-analysis: simulation methods as good alternatives to trend and direct method estimation. , 2017, Journal of clinical epidemiology.