Meta-analysis of test accuracy studies with multiple and missing thresholds : a multivariate-normal model

Background: When meta-analysing studies examining the diagnostic/predictive accuracy of classifications based on a continuous test, each study may provide results for one or more thresholds, which can vary across studies. Researchers typically meta-analyse each threshold independently. We consider a multivariate meta-analysis to synthesise results for all thresholds simultaneously and account for their correlation. Methods: We assume that the logit sensitivity and logit specificity estimates follow a multivariate-normal distribution within studies. We model the true logit sensitivity (logit specificity) as monotonically decreasing (increasing) functions of the continuous threshold. This produces a summary ROC curve, a summary estimate of sensitivity and specificity for each threshold, and reveals the heterogeneity in test accuracy across studies. Application is made to 13 studies of protein:creatinine ratio (PCR) for detecting significant proteinuria in pregnancy that each report up to nine thresholds, with 23 distinct thresholds across studies. Results: In the example there were large within-study and between-study correlations, which were accounted for by the method. A cubic relationship on the logit scale was a better fit for the summary ROC curve than a linear or quadratic one. Between-study heterogeneity was substantial. Based on the summary ROC curve, a PCR value of 0.30 to 0.35 corresponded to maximal pair of summary sensitivity and specificity. Limitations of the proposed model include the need to posit parametric functions for the relationship of sensitivity and specificity with the threshold, to ensure correct ordering of summary threshold results, and the multivariate-normal approximation to the within-study sampling distribution. Conclusion: The joint analysis of test performance data reported over multiple thresholds is feasible. The proposed approach handles different sets of available thresholds per study, and produces a summary ROC curve and summary results for each threshold to inform decision-making.

[1]  Bruce D. Spencer,et al.  The Delta Method , 1980 .

[2]  G. Lingman,et al.  Fetal and uteroplacental haemodynamics during short‐term atenolol treatment of hypertension in pregnancy , 1987, British journal of obstetrics and gynaecology.

[3]  C B Begg,et al.  A General Regression Methodology for ROC Curve Estimation , 1988, Medical decision making : an international journal of the Society for Medical Decision Making.

[4]  Geographic variation in the incidence of hypertension in pregnancy. World Health Organization International Collaborative Study of Hypertensive Disorders of Pregnancy. , 1988, American journal of obstetrics and gynecology.

[5]  L E Moses,et al.  Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. , 1993, Statistics in medicine.

[6]  L E Moses,et al.  Estimating Diagnostic Accuracy from Multiple Conflicting Reports , 1993, Medical decision making : an international journal of the Society for Medical Decision Making.

[7]  Frederick Mosteller,et al.  Guidelines for Meta-analyses Evaluating Diagnostic Tests , 1994, Annals of Internal Medicine.

[8]  F. Buntinx,et al.  Meta-analysis of ROC Curves , 2000, Medical decision making : an international journal of the Society for Medical Decision Making.

[9]  Jonathan J Deeks,et al.  Systematic reviews in health care: Systematic reviews of evaluations of diagnostic and screening tests. , 2001, BMJ.

[10]  C M Rutter,et al.  A hierarchical regression approach to meta‐analysis of diagnostic test accuracy evaluations , 2001, Statistics in medicine.

[11]  Jean-Marie Moutquin,et al.  The Classification and Diagnosis of the Hypertensive Disorders of Pregnancy: Statement from the International Society for the Study of Hypertension in Pregnancy (ISSHP) , 2001, Hypertension in pregnancy.

[12]  N. Higham.  Computing the nearest correlation matrix—a problem from finance , 2002 .

[13]  Kerrie Mengersen,et al.  Multivariate meta‐analysis , 2003, Statistics in medicine.

[14]  C Gatsonis,et al.  Meta‐analysis of Diagnostic Test Accuracy Assessment Studies with Varying Number of Thresholds , 2003, Biometrics.

[15]  Petra Macaskill,et al.  Empirical Bayes estimates generated in a hierarchical summary ROC analysis agreed closely with those of a full Bayesian analysis. , 2004, Journal of clinical epidemiology.

[16]  Wai-Yin Poon,et al.  A latent normal distribution model for analysing ordinal responses with applications in meta‐analysis , 2004, Statistics in medicine.

[17]  Johannes B Reitsma,et al.  Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. , 2005, Journal of clinical epidemiology.

[18]  Haitao Chu,et al.  Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. , 2006, Journal of clinical epidemiology.

[19]  A Metin Gülmezoglu,et al.  WHO analysis of causes of maternal death: a systematic review , 2006, The Lancet.

[20]  Jaap Stoker,et al.  Multivariate random-effects approach: for meta-analysis of cancer staging studies. , 2007, Academic radiology.

[21]  Roger M Harbord,et al.  A unification of models for meta-analysis of diagnostic accuracy studies. , 2007, Biostatistics.

[22]  P C Lambert,et al.  An evaluation of bivariate random‐effects meta‐analysis for the joint synthesis of two correlated outcomes , 2007, Statistics in medicine.

[23]  Cemach.  Saving mothers' lives: reviewing maternal deaths to make motherhood safer - 2003-2005 , 2007 .

[24]  Theo Stijnen,et al.  The binomial distribution of meta-analysis was preferred to model within-study variability. , 2008, Journal of clinical epidemiology.

[25]  Theo Stijnen,et al.  Multivariate random effects meta-analysis of diagnostic tests with multiple thresholds , 2009, BMC medical research methodology.

[26]  R Mann,et al.  Methods to identify postnatal depression in primary care: an integrated evidence synthesis and value of information analysis. , 2009, Health technology assessment.

[27]  Anne Whitehead,et al.  Meta-analysis of individual patient data versus aggregate data from longitudinal clinical trials , 2009, Clinical trials.

[28]  Richard D Riley,et al.  Multivariate meta‐analysis: the effect of ignoring within‐study correlation , 2009 .

[29]  David J Spiegelhalter,et al.  A re-evaluation of random-effects meta-analysis , 2009, Journal of the Royal Statistical Society. Series A,.

[30]  Theo Stijnen,et al.  Random effects meta‐analysis of event outcome in the framework of the generalized linear mixed model with applications in sparse data , 2010, Statistics in medicine.

[31]  H Putter,et al.  Meta‐Analysis of Diagnostic Test Accuracy Studies with Multiple Thresholds using Survival Methods , 2009, Biometrical journal. Biometrische Zeitschrift.

[32]  Haitao Chu,et al.  Bivariate Random Effects Meta-Analysis of Diagnostic Studies Using Generalized Linear Mixed Models , 2010, Medical decision making : an international journal of the Society for Medical Decision Making.

[33]  Dan Jackson,et al.  Multivariate meta-analysis: Potential and promise , 2011, Statistics in medicine.

[34]  R. Riley,et al.  Diagnostic accuracy of spot urinary protein and albumin to creatinine ratios for detection of significant proteinuria or adverse pregnancy outcome in patients with suspected pre-eclampsia: systematic review and meta-analysis , 2012, BMJ : British Medical Journal.

[35]  Richard D Riley,et al.  A multivariate meta‐analysis approach for reducing the impact of outcome reporting bias in systematic reviews , 2012, Statistics in medicine.

[36]  Yemisi Takwoingi,et al.  Empirical Evidence of the Importance of Comparative Studies of Diagnostic Test Accuracy , 2013, Annals of Internal Medicine.