Integration of Meta-analysis and Economic Decision Modeling for Evaluating Diagnostic Tests

Meta-analysis of diagnostic test accuracy data is more difficult than of effectiveness data because of 1) statistical challenges of dealing with multiple measures of accuracy (e.g., sensitivity and specificity) simultaneously and 2) incorporating threshold effects. A number of meta-analysis models are in use, ranging from naïve synthesis of independent sensitivity and specificity to optimization of a hierarchical summary receiver operating characteristic (SROC) curve. Little work has been done on how such analyses should inform decision models. This article aims to present a unified framework for the synthesis of primary data and economic evaluation of alternative diagnostic testing strategies using Bayesian Markov Chain Monte Carlo simulation methods. The authors extend this previous work by using systematic review to derive model parameters, fully allowing for uncertainty in their estimation, and formally incorporating variability between study results into the decision analysis. Using a simple decision model comparing alternative testing strategies for suspected deep vein thrombosis as an example, the authors consider how to use outputs of different alternative meta-analysis models in decision models. They also explore the limitations of diagnostic test studies, particularly when there is no obvious threshold value. To correct some of the limitations of diagnostic test studies, they propose that tests with implicit and explicit thresholds should be studied using distinctly different frameworks. Specifically, when a threshold exists, quantitative threshold information should be included in meta-analysis models to aid interpretation of SROCs. Setting policy to relate to a specific point may be much more difficult for studies with implicit thresholds.

[1]  A E Ades,et al.  The Interpretation of Random-Effects Meta-Analysis in Decision Models , 2005, Medical decision making : an international journal of the Society for Medical Decision Making.

[2]  Steve Goodacre,et al.  Meta-Analysis: The Value of Clinical Assessment in the Diagnosis of Deep Venous Thrombosis , 2005, Annals of Internal Medicine.

[3]  Johannes B Reitsma,et al.  Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. , 2005, Journal of clinical epidemiology.

[4]  J. C. Houwelingen,et al.  Bivariate Random Effects Meta-Analysis of ROC Curves , 2008, Medical decision making : an international journal of the Society for Medical Decision Making.

[5]  F. Buntinx,et al.  Meta-analysis of ROC Curves , 2000, Medical decision making : an international journal of the Society for Medical Decision Making.

[6]  Alex J. Sutton,et al.  A Bayesian approach to Markov modelling in cost‐effectiveness analyses: application to taxane use in advanced breast cancer , 2003 .

[7]  C M Rutter,et al.  A hierarchical regression approach to meta‐analysis of diagnostic test accuracy evaluations , 2001, Statistics in medicine.

[8]  R A Greenes,et al.  Patient-Oriented Performance Measures of Diagnostic Tests , 1984, Medical decision making : an international journal of the Society for Medical Decision Making.

[9]  A E Ades,et al.  Expected Value of Sample Information Calculations in Medical Decision Modeling , 2004, Medical decision making : an international journal of the Society for Medical Decision Making.

[10]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[11]  Jonathan J Deeks,et al.  Systematic reviews in health care: Systematic reviews of evaluations of diagnostic and screening tests. , 2001, BMJ.

[12]  Alastair Fischer,et al.  The economics of diagnosis. , 2006, Health economics.

[13]  Paul Glasziou,et al.  Comparative accuracy: assessing new tests against existing diagnostic pathways , 2006, BMJ : British Medical Journal.

[14]  M. Kohn,et al.  What White Blood Cell Count Should Prompt Antibiotic Treatment in a Febrile Child? Tutorial on the Importance of Disease Likelihood to the Interpretation of Diagnostic Tests , 2001, Medical decision making : an international journal of the Society for Medical Decision Making.

[15]  Lucas M Bachmann,et al.  Systematic reviews with individual patient data meta-analysis to evaluate diagnostic tests. , 2003, European journal of obstetrics, gynecology, and reproductive biology.

[16]  T Stijnen,et al.  Baseline risk as predictor of treatment benefit: three clinical meta-re-analyses. , 2000, Statistics in medicine.

[17]  Alvin I Mushlin,et al.  Costeffectiveness of diagnostic tests , 2001, The Lancet.

[18]  David Turner,et al.  Comprehensive decision analytical modelling in economic evaluation: a Bayesian approach. , 2004, Health economics.

[19]  A J Sutton,et al.  Variation in the diagnostic performance of D-dimer for suspected deep vein thrombosis. , 2005, QJM : monthly journal of the Association of Physicians.

[20]  Steve Goodacre,et al.  Bmc Medical Imaging Systematic Review and Meta-analysis of the Diagnostic Accuracy of Ultrasonography for Deep Vein Thrombosis , 2005 .

[21]  David R. Jones,et al.  How vague is vague? A simulation study of the impact of the use of vague prior distributions in MCMC using WinBUGS , 2005, Statistics in medicine.

[22]  M. Weinstein,et al.  Decision Making in Health and Medicine , 2001 .

[23]  Patrick M M Bossuyt,et al.  Exploring sources of heterogeneity in systematic reviews of diagnostic tests , 2002, Statistics in medicine.

[24]  S G Thompson,et al.  A likelihood approach to meta-analysis with random effects. , 1996, Statistics in medicine.

[25]  Mark Stevenson,et al.  Measurement of the clinical and cost-effectiveness of non-invasive diagnostic testing strategies for deep vein thrombosis. , 2006, Health technology assessment.

[26]  David J Spiegelhalter,et al.  Bayesian approaches to multiple sources of evidence and uncertainty in complex cost‐effectiveness modelling , 2003, Statistics in medicine.

[27]  John A. Swets,et al.  Signal Detection Theory and ROC Analysis in Psychology and Diagnostics: Collected Papers , 1996 .

[28]  M. Sculpher,et al.  Representing uncertainty: the role of cost-effectiveness acceptability curves. , 2001, Health economics.

[29]  P. Bossuyt,et al.  BMC Medical Research Methodology , 2002 .

[30]  P Doubilet,et al.  A Mathematical Approach to Interpretation and Selection of Diagnostic Tests , 1983, Medical decision making : an international journal of the Society for Medical Decision Making.

[31]  C Gatsonis,et al.  Meta‐analysis of Diagnostic Test Accuracy Assessment Studies with Varying Number of Thresholds , 2003, Biometrics.

[32]  David J Samson,et al.  Challenges in Systematic Reviews of Diagnostic Technologies , 2005, Annals of Internal Medicine.

[33]  A J Sutton,et al.  How should we diagnose suspected deep-vein thrombosis? , 2006, QJM : monthly journal of the Association of Physicians.

[34]  John Mullahy,et al.  Net Health Benefits: A New Framework for the Analysis of Uncertainty in Cost-Effectiveness Analysis , 1998 .

[35]  P Glasziou,et al.  Test Selection Measures , 1989, Medical decision making : an international journal of the Society for Medical Decision Making.

[36]  Haitao Chu,et al.  A unification of models for meta-analysis of diagnostic accuracy studies. , 2009, Biostatistics.

[37]  Martine Wallon,et al.  Methods to Estimate the Optimal Threshold for Normally or Log-Normally Distributed Biological Tests , 2005, Medical decision making : an international journal of the Society for Medical Decision Making.

[38]  Alastair Fischer,et al.  Non-linearity in the cost-effectiveness frontier. , 2006, Health economics.

[39]  A E Ades,et al.  Correction: Interpretation of Random Effects Meta-analysis in Decision Models , 2007, Medical decision making : an international journal of the Society for Medical Decision Making.

[40]  Bas Groot Koerkamp,et al.  Limitations of Acceptability Curves for Presenting Uncertainty in Cost-Effectiveness Analysis , 2007, Medical decision making : an international journal of the Society for Medical Decision Making.

[41]  A. Mushlin,et al.  Focusing Technology Assessment Using Medical Decision Theory , 1988, Medical decision making : an international journal of the Society for Medical Decision Making.

[42]  G. ter Riet,et al.  Systematic reviews of evaluations of diagnostic and screening tests , 2001, BMJ : British Medical Journal.

[43]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .