Multivariate random effects meta-analysis of diagnostic tests with multiple thresholds

BackgroundBivariate random effects meta-analysis of diagnostic tests is becoming a well established approach when studies present one two-by-two table or one pair of sensitivity and specificity. When studies present multiple thresholds for test positivity, usually meta-analysts reduce the data to a two-by-two table or take one threshold value at a time and apply the well developed meta-analytic approaches. However, this approach does not fully exploit the data.MethodsIn this paper we generalize the bivariate random effects approach to the situation where test results are presented with k thresholds for test positivity, resulting in a 2 by (k+1) table per study. The model can be fitted with standard likelihood procedures in statistical packages such as SAS (Proc NLMIXED). We follow a multivariate random effects approach; i.e., we assume that each study estimates a study specific ROC curve that can be viewed as randomly sampled from the population of all ROC curves of such studies. In contrast to the bivariate case, where nothing can be said about the shape of study specific ROC curves without additional untestable assumptions, the multivariate model can be used to describe study specific ROC curves. The models are easily extended with study level covariates.ResultsThe method is illustrated using published meta-analysis data. The SAS NLMIXED syntax is given in the appendix.ConclusionWe conclude that the multivariate random effects meta-analysis approach is an appropriate and convenient framework to meta-analyse studies with multiple threshold without losing any information by dichotomizing the test results.

[1]  Johannes B Reitsma,et al.  Meta-Analysis of Diagnostic Studies: A Comparison of Random Intercept, Normal-Normal, and Binomial-Normal Bivariate Summary ROC Approaches , 2008, Medical decision making : an international journal of the Society for Medical Decision Making.

[2]  Wai-Yin Poon,et al.  A latent normal distribution model for analysing ordinal responses with applications in meta‐analysis , 2004, Statistics in medicine.

[3]  Frederick Mosteller,et al.  Guidelines for Meta-analyses Evaluating Diagnostic Tests , 1994, Annals of Internal Medicine.

[4]  B Littenberg,et al.  A Meta-analytic Method for Summarizing Diagnostic Test Performances , 1993, Medical decision making : an international journal of the Society for Medical Decision Making.

[5]  C Gatsonis,et al.  Meta‐analysis of Diagnostic Test Accuracy Assessment Studies with Varying Number of Thresholds , 2003, Biometrics.

[6]  L E Moses,et al.  Estimating Diagnostic Accuracy from Multiple Conflicting Reports , 1993, Medical decision making : an international journal of the Society for Medical Decision Making.

[7]  L E Moses,et al.  Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. , 1993, Statistics in medicine.

[8]  C M Rutter,et al.  A hierarchical regression approach to meta‐analysis of diagnostic test accuracy evaluations , 2001, Statistics in medicine.

[9]  Bruce D. Spencer,et al.  The Delta Method , 1980 .

[10]  Haitao Chu,et al.  Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. , 2006, Journal of clinical epidemiology.

[11]  Roger M Harbord,et al.  A unification of models for meta-analysis of diagnostic accuracy studies. , 2007, Biostatistics.

[12]  Johannes B Reitsma,et al.  Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. , 2005, Journal of clinical epidemiology.

[13]  F. Buntinx,et al.  Meta-analysis of ROC Curves , 2000, Medical decision making : an international journal of the Society for Medical Decision Making.

[14]  J. C. Houwelingen,et al.  Bivariate Random Effects Meta-Analysis of ROC Curves , 2008, Medical decision making : an international journal of the Society for Medical Decision Making.

[15]  S. Merhar,et al.  Letter to the editor , 2005, IEEE Communications Magazine.

[16]  P Glasziou,et al.  Meta-analytic methods for diagnostic test accuracy. , 1995, Journal of clinical epidemiology.

[17]  Theo Stijnen,et al.  The binomial distribution of meta-analysis was preferred to model within-study variability. , 2008, Journal of clinical epidemiology.

[18]  N. Laird,et al.  Meta-analysis in clinical trials. , 1986, Controlled clinical trials.

[19]  F. Buntinx,et al.  The value of the CAGE in screening for alcohol abuse and alcohol dependence in general clinical populations: a diagnostic meta-analysis. , 2004, Journal of clinical epidemiology.

[20]  S G Thompson,et al.  Investigating underlying risk as a source of heterogeneity in meta-analysis. , 1997, Statistics in medicine.

[21]  H. V. van Houwelingen,et al.  Investigating underlying risk as a source of heterogeneity in meta-analysis. , 1999, Statistics in medicine.

[22]  T P Speed,et al.  A score test for the linkage analysis of qualitative and quantitative traits based on identity by descent data from sib-pairs. , 2000, Biostatistics.

[23]  C B Begg,et al.  A General Regression Methodology for ROC Curve Estimation , 1988, Medical decision making : an international journal of the Society for Medical Decision Making.

[24]  Jaap Stoker,et al.  Multivariate random-effects approach: for meta-analysis of cancer staging studies. , 2007, Academic radiology.

[25]  Theo Stijnen,et al.  Advanced methods in meta‐analysis: multivariate approach and meta‐regression , 2002, Statistics in medicine.

[26]  J. Hermans,et al.  The value of aspiration cytologic examination of the breast a statistical review of the medical literature , 1992, Cancer.

[27]  A. I. Mushlin,et al.  Diagnostic tests in breast cancer. Clinical strategies based on diagnostic probabilities. , 1985, Annals of internal medicine.

[28]  Ulrike Dapp,et al.  Development, feasibility and performance of a health risk appraisal questionnaire for older persons , 2007, BMC medical research methodology.

[29]  Alan Agresti,et al.  Categorical Data Analysis , 2003 .

[30]  G. Molenberghs,et al.  Linear Mixed Models for Longitudinal Data , 2001 .

[31]  G. Molenberghs,et al.  Models for Discrete Longitudinal Data , 2005 .

[32]  Dafydd Gibbon,et al.  1 User’s guide , 1998 .

[33]  Ana Ivelisse Avilés,et al.  Linear Mixed Models for Longitudinal Data , 2001, Technometrics.

[34]  Richard D Riley,et al.  Beyond the Bench: Hunting Down Fugitive Literature , 2004, Environmental Health Perspectives.

[35]  Haitao Chu,et al.  A unification of models for meta-analysis of diagnostic accuracy studies. , 2009, Biostatistics.

[36]  Theo Stijnen,et al.  Associating explanatory variables with summary receiver operating characteristic curves in diagnostic meta-analysis. , 2009, Journal of clinical epidemiology.