Reducing cost in sequential testing: a limit of indifference approach

In noninferiority studies, a limit of indifference is used to express a tolerance in results such that the clinician would regard such results as being acceptable or ‘not worse’. We applied this concept to a measure of accuracy, the Receiver Operating Characteristic (ROC) curve, for a sequence of tests. We expressed a limit of indifference for the range of acceptable sensitivity values and examined the associated cost of testing within this range. In doing so, we generated the minimum cost maximum ROC (MCMROC) curve, which reflects the reduced sensitivity and cost of testing. We compared the MCMROC and its associated cost curve between limits of indifference set to 0.999 [a 0.1% reduction in true positive rate (TPR)], 0.95 (a 5% reduction in TPR), and 1 (no reduction in TPR). The limit of indifference tended to have less of an effect on the MCMROC curves than on the associated cost curves that were greatly affected. Cost was reduced at high false positive rates (FPRs) at higher limit of indifference (0.999) and at small FPRs as the limit of indifference decreased (0.95). These patterns were also observed as applied to sequential strategies used to diagnose diabetes in the Pima Indians. Copyright © 2013 John Wiley & Sons, Ltd.

[1]  K. Zou,et al.  Statistical combination schemes of repeated diagnostic test data. , 2006, Academic radiology.

[2]  Mary Lou Thompson,et al.  Assessing the diagnostic accuracy of a sequence of tests. , 2003, Biostatistics.

[3]  D. McClish,et al.  Accuracy and cost comparison in medical testing using sequential testing strategies , 2011, Statistics in medicine.

[4]  Les Irwig,et al.  Assessing the gain in diagnostic performance when combining two diagnostic tests , 2002, Statistics in medicine.

[5]  J A Bean,et al.  On the sample size for one-sided equivalence of sensitivities based upon McNemar's test. , 1995, Statistics in medicine.

[6]  Wesley O Johnson,et al.  Diagnostic test accuracy and prevalence inferences based on joint and sequential testing with finite population sampling , 2004, Statistics in medicine.

[7]  W. Cumberland,et al.  Sample size determination for equivalence test using rate ratio of sensitivity and specificity in paired sample data. , 2001, Controlled clinical trials.

[8]  R. Marshall,et al.  The Predictive Value of Simple Rules for Combining Two Diagnostic Tests , 1989 .

[9]  Gary K Grunwald,et al.  Estimates of sensitivity and specificity can be biased when reporting the results of the second test in a screening trial conducted in series , 2010, BMC medical research methodology.

[10]  Ingo Ruczinski,et al.  Logic Regression — Methods and Software , 2003 .

[11]  Richard S. Johannes,et al.  Using the ADAP Learning Algorithm to Forecast the Onset of Diabetes Mellitus , 1988 .

[12]  S G Baker,et al.  Identifying Combinations of Cancer Markers for Further Study as Triggers of Early Intervention , 2000, Biometrics.

[13]  T S Lau,et al.  On dependent repeated screening tests. , 1991, Biometrics.

[14]  Hua Jin,et al.  A Procedure for Determining Whether a Simple Combination of Diagnostic Tests May Be Noninferior to the Theoretical Optimum Combination , 2008, Medical decision making : an international journal of the Society for Medical Decision Making.

[15]  Catherine Blake,et al.  UCI Repository of machine learning databases , 1998 .

[16]  Harry K Genant,et al.  On the non‐inferiority of a diagnostic test based on paired observations , 2003, Statistics in medicine.

[17]  J. Reed,et al.  Utility of combining two diagnostic tests. , 1991, Computer methods and programs in biomedicine.

[18]  E. Bixler,et al.  Modelling population heterogeneity in sensitivity and specificity of a multi-stage screen for obstructive sleep apnoea. , 1997, Statistics in medicine.

[19]  S Geisser,et al.  Optimal administration of dual screening tests for detecting a characteristic with special reference to low prevalence diseases. , 1992, Biometrics.