Incorporating published univariable associations in diagnostic and prognostic modeling

BackgroundDiagnostic and prognostic literature is overwhelmed with studies reporting univariable predictor-outcome associations. Currently, methods to incorporate such information in the construction of a prediction model are underdeveloped and unfamiliar to many researchers.MethodsThis article aims to improve upon an adaptation method originally proposed by Greenland (1987) and Steyerberg (2000) to incorporate previously published univariable associations in the construction of a novel prediction model. The proposed method improves upon the variance estimation component by reconfiguring the adaptation process in established theory and making it more robust. Different variants of the proposed method were tested in a simulation study, where performance was measured by comparing estimated associations with their predefined values according to the Mean Squared Error and coverage of the 90% confidence intervals.ResultsResults demonstrate that performance of estimated multivariable associations considerably improves for small datasets where external evidence is included. Although the error of estimated associations decreases with increasing amount of individual participant data, it does not disappear completely, even in very large datasets.ConclusionsThe proposed method to aggregate previously published univariable associations with individual participant data in the construction of a novel prediction models outperforms established approaches and is especially worthwhile when relatively limited individual participant data are available.

[1]  Patrick Royston,et al.  The design of simulation studies in medical statistics , 2006, Statistics in medicine.

[2]  A. Albert,et al.  On the existence of maximum likelihood estimates in logistic regression models , 1984 .

[3]  Karel G M Moons,et al.  Aggregating published prediction models with individual participant data: a comparison of different approaches , 2012, Statistics in medicine.

[4]  A. Evans,et al.  Translating Clinical Research into Clinical Practice: Impact of Using Prediction Rules To Make Decisions , 2006, Annals of Internal Medicine.

[5]  R. Riley,et al.  Meta-analysis of individual participant data: rationale, conduct, and reporting , 2010, BMJ : British Medical Journal.

[6]  Nicola J Cooper,et al.  Evidence synthesis as the key to more coherent and efficient research , 2009, BMC medical research methodology.

[7]  J. Habbema,et al.  Perioperative mortality of elective abdominal aortic aneurysm surgery. A clinical prediction rule based on literature and individual patient data. , 1995, Archives of internal medicine.

[8]  Charlotte H. Mason,et al.  Collinearity, power, and interpretation of multiple regression analysis. , 1991 .

[9]  C.J.H. Mann,et al.  Clinical Prediction Models: A Practical Approach to Development, Validation and Updating , 2009 .

[10]  S L Normand,et al.  Meta-analysis: formulating, evaluating, combining, and reporting. , 1999, Statistics in medicine.

[11]  P. Royston,et al.  Prognosis and prognostic research: application and impact of prognostic models in clinical practice , 2009, BMJ : British Medical Journal.

[12]  D. Altman,et al.  Where Next for Evidence Synthesis of Prognostic Marker Studies? Improving the Quality and Reporting of Primary Studies to Facilitate Clinically Relevant Evidence-Based Results , 2007 .

[13]  F. Harrell,et al.  Criteria for Evaluation of Novel Markers of Cardiovascular Risk: A Scientific Statement From the American Heart Association , 2009, Circulation.

[14]  M. G. Pittau,et al.  A weakly informative default prior distribution for logistic and other regression models , 2008, 0901.4011.

[15]  M. Woodward,et al.  Risk prediction models: II. External validation, model updating, and impact assessment , 2012, Heart.

[16]  Emmanuel Lesaffre,et al.  Partial Separation in Logistic Discrimination , 1989 .

[17]  Richard D Riley,et al.  Evidence synthesis combining individual patient data and aggregate data: a systematic review identified current practice and possible methods. , 2007, Journal of clinical epidemiology.

[18]  N. Jewell,et al.  Some surprising results about covariate adjustment in logistic regression models , 1991 .

[19]  J. Auget,et al.  Advances in statistical methods for the health sciences : applications to cancer and AIDS studies, genome sequence analysis, and survival analysis , 2007 .

[20]  S Greenland,et al.  Quantitative methods in the review of epidemiologic literature. , 1987, Epidemiologic reviews.

[21]  J. Concato,et al.  A simulation study of the number of events per variable in logistic regression analysis. , 1996, Journal of clinical epidemiology.

[22]  H. Sox,et al.  Clinical prediction rules. Applications and methodological standards. , 1985, The New England journal of medicine.

[23]  T. Alonzo Clinical Prediction Models: A Practical Approach to Development, Validation, and Updating By Ewout W. Steyerberg , 2009 .

[24]  Richard D Riley,et al.  Evidence-Based Assessment and Application of Prognostic Markers: The Long Way from Single Studies to Meta-Analysis , 2006 .

[25]  S Greenland,et al.  Invited commentary: a critical look at some popular meta-analytic methods. , 1994, American journal of epidemiology.

[26]  L. Stewart,et al.  To IPD or not to IPD? , 2002, Evaluation & the health professions.

[27]  E W Steyerberg,et al.  See Blockindiscussions, Blockinstats, Blockinand Blockinauthor Blockinprofiles Blockinfor Blockinthis Blockinpublication Prognostic Blockinmodels Blockinbased Blockinon Blockinliterature Blockinand Individual Blockinpatient Blockindata Blockinin Blockinlogistic Blockinregression Analysis Article Blo , 2022 .

[28]  K. Moons Criteria for scientific evaluation of novel markers: a perspective. , 2010, Clinical chemistry.

[29]  John P A Ioannidis,et al.  Commentary: meta-analysis of individual participants' data in genetic epidemiology. , 2002, American journal of epidemiology.

[30]  D G Altman,et al.  Prognostic markers in cancer: the evolution of evidence from single studies to meta-analysis, and beyond , 2009, British Journal of Cancer.

[31]  D. Bennett Review of analytical methods for prospective cohort studies using time to event data: single studies and implications for meta-analysis , 2003, Statistical methods in medical research.

[32]  Mike Clarke,et al.  Doing New Research? Don't Forget the Old , 2004, PLoS medicine.

[33]  M. Woodward,et al.  Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker , 2012, Heart.

[34]  L. Hedges,et al.  Fixed- and random-effects models in meta-analysis. , 1998 .

[35]  E. Steyerberg Clinical Prediction Models , 2008, Statistics for Biology and Health.

[36]  D. Altman,et al.  Measuring inconsistency in meta-analyses , 2003, BMJ : British Medical Journal.

[37]  L. Stewart,et al.  Practical methodology of meta-analyses (overviews) using updated individual patient data. Cochrane Working Group. , 1995, Statistics in medicine.

[38]  Sander Greenland,et al.  Closed form and dually consistent methods for inference on strict collapsibility in 2×2×K and 2×J×K tables , 1988 .