Meta‐analysis of diagnostic test studies using individual patient data and aggregate data

A meta-analysis of diagnostic test studies provides evidence-based results regarding the accuracy of a particular test, and usually involves synthesizing aggregate data (AD) from each study, such as the 2 by 2 tables of diagnostic accuracy. A bivariate random-effects meta-analysis (BRMA) can appropriately synthesize these tables, and leads to clinical results, such as the summary sensitivity and specificity across studies. However, translating such results into practice may be limited by between-study heterogeneity and that they relate to some 'average' patient across studies.In this paper we describe how the meta-analysis of individual patient data (IPD) from diagnostic studies can lead to clinical results more tailored to the individual patient. We develop IPD models that extend the BRMA framework to include study-level covariates, which help explain the between-study heterogeneity, and also patient-level covariates, which allow one to assess the effect of patient characteristics on test accuracy. We show how the inclusion of patient-level covariates requires a careful separation of within-study and across-study accuracy-covariate effects, as the latter are particularly prone to confounding. Our models are assessed through simulation and extended to allow IPD studies to be combined with AD studies, as IPD are not always available for all studies. Application is made to 23 studies assessing the accuracy of ear thermometers for diagnosing fever in children, with 16 IPD and 7 AD studies. The models reveal that between-study heterogeneity is partly explained by the use of different measurement devices, but there is no evidence that being an infant modifies diagnostic accuracy.

[1]  P. Albert,et al.  Models for longitudinal data: a generalized estimating equation approach. , 1988, Biometrics.

[2]  J. Kalbfleisch,et al.  Between- and within-cluster covariate effects in the analysis of clustered data. , 1998, Biometrics.

[3]  H Goldstein,et al.  A multilevel model framework for meta-analysis of clinical trials with binary outcomes. , 2000, Statistics in medicine.

[4]  K R Abrams,et al.  Bayesian Approaches to Meta-analysi of ROC Curves , 1999, Medical decision making : an international journal of the Society for Medical Decision Making.

[5]  Sylvia Richardson,et al.  Improving ecological inference using individual‐level data , 2006, Statistics in medicine.

[6]  Christopher H Schmid,et al.  Summing up evidence: one answer is not always enough , 1998, The Lancet.

[7]  Patrick M Bossuyt,et al.  We should not pool diagnostic likelihood ratios in systematic reviews , 2008, Statistics in medicine.

[8]  Harold I Feldman,et al.  Individual patient‐ versus group‐level data meta‐regressions for the investigation of treatment effect modifiers: ecological bias rears its ugly head , 2002, Statistics in medicine.

[9]  F. Buntinx,et al.  Meta-analysis of ROC Curves , 2000, Medical decision making : an international journal of the Society for Medical Decision Making.

[10]  Paul Landais,et al.  Meta-regression detected associations between heterogeneous treatment effects and study-level, but not patient-level, factors. , 2004, Journal of clinical epidemiology.

[11]  L E Moses,et al.  Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. , 1993, Statistics in medicine.

[12]  Lucas M Bachmann,et al.  Systematic reviews with individual patient data meta-analysis to evaluate diagnostic tests. , 2003, European journal of obstetrics, gynecology, and reproductive biology.

[13]  D. Bates,et al.  Approximations to the Log-Likelihood Function in the Nonlinear Mixed-Effects Model , 1995 .

[14]  S G Thompson,et al.  Systematic Review: Why sources of heterogeneity in meta-analysis should be investigated , 1994, BMJ.

[15]  Michael K Parides,et al.  Separation of individual‐level and cluster‐level covariate effects in regression analysis of correlated data , 2003, Statistics in medicine.

[16]  Roger M Harbord,et al.  A unification of models for meta-analysis of diagnostic accuracy studies. , 2007, Biostatistics.

[17]  P Glasziou,et al.  Meta-analytic methods for diagnostic test accuracy. , 1995, Journal of clinical epidemiology.

[18]  Jonathan J Deeks,et al.  Systematic reviews in health care: Systematic reviews of evaluations of diagnostic and screening tests. , 2001, BMJ.

[19]  R. Carroll,et al.  Measurement error, instrumental variables and corrections for attenuation with applications to meta-analyses. , 1994, Statistics in medicine.

[20]  David R. Jones,et al.  How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS , 2005, Statistics in medicine.

[21]  Richard D Riley,et al.  Evidence synthesis combining individual patient data and aggregate data: a systematic review identified current practice and possible methods. , 2007, Journal of clinical epidemiology.

[22]  A J Sutton,et al.  Meta‐analysis of individual‐ and aggregate‐level data , 2008, Statistics in medicine.

[23]  A. Gelman.  Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper) , 2004 .

[24]  William J. Browne,et al.  Implementation and performance issues in the Bayesian and likelihood fitting of multilevel models , 2000, Comput. Stat..

[25]  M C Simmonds,et al.  Covariate heterogeneity in meta‐analysis: Criteria for deciding between meta‐regression and individual patient data , 2007, Statistics in medicine.

[26]  J Shepherd,et al.  Clinical effectiveness and cost-effectiveness of drotrecogin alfa (activated) (Xigris) for the treatment of severe sepsis in adults: a systematic review and economic evaluation. , 2005, Health technology assessment.

[27]  Johannes B Reitsma,et al.  Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. , 2005, Journal of clinical epidemiology.

[28]  Simon G Thompson,et al.  Flexible parametric models for random‐effects distributions , 2008, Statistics in medicine.

[29]  Petra Macaskill,et al.  Empirical Bayes estimates generated in a hierarchical summary ROC analysis agreed closely with those of a full Bayesian analysis. , 2004, Journal of clinical epidemiology.

[30]  A Whitehead,et al.  Meta‐analysis of continuous outcome data from individual patients , 2001, Statistics in medicine.

[31]  S Greenland,et al.  Ecological bias, confounding, and effect modification. , 1989, International journal of epidemiology.

[32]  J. Kalbfleisch,et al.  A Comparison of Cluster-Specific and Population-Averaged Approaches for Analyzing Correlated Binary Data , 1991 .

[33]  Theo Stijnen,et al.  The binomial distribution of meta-analysis was preferred to model within-study variability. , 2008, Journal of clinical epidemiology.

[34]  Richard D Riley,et al.  Beyond the Bench: Hunting Down Fugitive Literature , 2004, Environmental Health Perspectives.

[35]  Theo Stijnen,et al.  Advanced methods in meta‐analysis: multivariate approach and meta‐regression , 2002, Statistics in medicine.

[36]  J. Deeks,et al.  A methodological review of how heterogeneity has been examined in systematic reviews of diagnostic test accuracy. , 2005, Health technology assessment.

[37]  S. Richardson,et al.  Hierarchical related regression for combining aggregate and individual data in studies of socio‐economic disease risk factors , 2007 .

[38]  C S Berkey,et al.  A random-effects regression model for meta-analysis. , 1995, Statistics in medicine.

[39]  P. Williamson,et al.  In a systematic review, infrared ear thermometry for fever diagnosis in children finds poor sensitivity. , 2006, Journal of clinical epidemiology.

[40]  Paula R Williamson,et al.  Investigating heterogeneity in an individual patient data meta‐analysis of time to event outcomes , 2005, Statistics in medicine.

[41]  P. Bossuyt,et al.  Sources of Variation and Bias in Studies of Diagnostic Accuracy , 2004, Annals of Internal Medicine.

[42]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[43]  Richard D Riley,et al.  Meta‐analysis of continuous outcomes combining individual patient data and aggregate data , 2008, Statistics in medicine.

[44]  C M Rutter,et al.  A hierarchical regression approach to meta‐analysis of diagnostic test accuracy evaluations , 2001, Statistics in medicine.

[45]  Patrick M M Bossuyt,et al.  Exploring sources of heterogeneity in systematic reviews of diagnostic tests , 2002, Statistics in medicine.

[46]  Simon G Thompson,et al.  Can meta-analysis help target interventions at individuals most likely to benefit? , 2005, The Lancet.

[47]  P. Williamson,et al.  Meta‐analysis of method comparison studies , 2002, Statistics in medicine.

[48]  Paula R Williamson,et al.  Infrared ear thermometry compared with rectal thermometry in children: a systematic review , 2002, The Lancet.

[49]  F Mosteller,et al.  Meta-analysis of multiple outcomes by regression with random effects. , 1998, Statistics in medicine.

[50]  Haitao Chu,et al.  Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. , 2006, Journal of clinical epidemiology.

[51]  P C Lambert,et al.  A comparison of summary patient-level covariates in meta-regression with individual patient data meta-analysis. , 2002, Journal of clinical epidemiology.