A Procedure for Determining Whether a Simple Combination of Diagnostic Tests May Be Noninferior to the Theoretical Optimum Combination

Diagnosis accuracy can be improved by combining several complementary diagnostic tests. The likelihood ratio (LR) function is the optimal combination because it achieves the largest area under the receiver operating characteristic (ROC) curve. Derivation and interpretation of the LR function, however, can be complicated. Linear discriminant/logistic functions are simple but may not be optimal. In this article, the authors propose a statistical framework to determine when such linear combinations are noninferior alternatives to the LR function. They propose a nonparametric procedure to calculate LR functions and estimate the optimal area under the curve (AUC). The authors then define noninferiority of a simpler combination to the LR function with regard to the AUC of ROC curves. A bootstrap procedure is proposed to test the noninferiority. Monte Carlo simulation experiments are used to evaluate the performance of the proposed methods. Data from the Study of Osteoporotic Fractures are used to demonstrate the procedure.

[1]  Margaret Sullivan Pepe,et al.  Combining Several Screening Tests: Optimality of the Risk Score , 2002, Biometrics.

[2]  Jun S. Liu,et al.  Linear Combinations of Multiple Diagnostic Markers , 1993 .

[3]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .

[4]  H. Genant,et al.  Classification Algorithms for Hip Fracture Prediction Based on Recursive Partitioning Methods , 2004, Medical decision making : an international journal of the Society for Medical Decision Making.

[5]  A. Izenman.  Review Papers: Recent Developments in Nonparametric Density Estimation , 1991 .

[6]  M. C. Jones,et al.  Comparison of Smoothing Parameterizations in Bivariate Kernel Density Estimation , 1993 .

[7]  E. DeLong,et al.  Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. , 1988, Biometrics.

[8]  A. Izenman.  Recent Developments in Nonparametric Density Estimation , 1991 .

[9]  Xiao-Hua Zhou,et al.  Statistical Methods in Diagnostic Medicine , 2002 .

[10]  C. Metz.  ROC Methodology in Radiologic Imaging , 1986, Investigative radiology.

[11]  M. Pepe,et al.  Combining diagnostic test results to increase accuracy. , 2000, Biostatistics.

[12]  J. Hanley.  Receiver operating characteristic (ROC) methodology: the state of the art. , 1989, Critical reviews in diagnostic imaging.

[13]  P. Hall.  Biometrika centenary: Nonparametrics , 2001 .

[14]  Robert Tibshirani,et al.  An Introduction to the Bootstrap , 1994 .

[15]  Nils Lid Hjort,et al.  Bayesian Approaches to Non- and Semiparametric Density Estimation , 1994 .

[16]  S. Cummings,et al.  Bone density at various sites for prediction of hip fractures , 1993, The Lancet.

[17]  Ü. Gürler,et al.  Bivariate Density Estimation with Randomly Truncated Data , 2000 .

[18]  S. Cummings,et al.  Biochemical Markers of Bone Turnover and Prediction of Hip Bone Loss in Older Women: The Study of Osteoporotic Fractures , 1999, Journal of bone and mineral research : the official journal of the American Society for Bone and Mineral Research.

[19]  S. Cummings,et al.  Endogenous hormones and the risk of hip and vertebral fractures among older women. Study of Osteoporotic Fractures Research Group. , 1998, The New England journal of medicine.

[20]  S. Cummings,et al.  Risk factors for hip fracture in white women. Study of Osteoporotic Fractures Research Group. , 1995, The New England journal of medicine.

[21]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[22]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[23]  N. Hjort,et al.  Nonparametric Density Estimation with a Parametric Start , 1995 .

[24]  S G Baker,et al.  Identifying Combinations of Cancer Markers for Further Study as Triggers of Early Intervention , 2000, Biometrics.

[25]  S. Cummings,et al.  Endogenous Hormones and the Risk of Hip and Vertebral Fractures Among Older Women , 1999 .

[26]  David W. Scott,et al.  Multivariate Density Estimation: Theory, Practice, and Visualization , 1992, Wiley Series in Probability and Statistics.

[27]  S. Cummings,et al.  Osteoarthritis and risk of falls, rates of bone loss, and osteoporotic fractures. Study of Osteoporotic Fractures Research Group. , 1999, Arthritis and rheumatism.

[28]  R. Randles,et al.  Generalized Linear and Quadratic Discriminant Functions Using Robust Estimates , 1978 .

[29]  Harry K Genant,et al.  On the non‐inferiority of a diagnostic test based on paired observations , 2003, Statistics in medicine.

[30]  S G Baker,et al.  Evaluating multiple diagnostic tests with partial verification. , 1995, Biometrics.

[31]  D. W. Scott,et al.  Multivariate Density Estimation, Theory, Practice and Visualization , 1992 .

[32]  R. Fisher.  THE USE OF MULTIPLE MEASUREMENTS IN TAXONOMIC PROBLEMS , 1936 .

[33]  Mary Lou Thompson,et al.  Assessing the diagnostic accuracy of a sequence of tests. , 2003, Biostatistics.

[34]  S. Cummings,et al.  A high ratio of dietary animal to vegetable protein increases the rate of bone loss and the risk of fracture in postmenopausal women. Study of Osteoporotic Fractures Research Group. , 2001, The American journal of clinical nutrition.

[35]  P A Lachenbruch,et al.  Assessing screening tests: extensions of McNemar's test. , 1998, Statistics in medicine.

[36]  M. Wand.  Error analysis for general multtvariate kernel estimators , 1992 .

[37]  M. C. Jones,et al.  Locally parametric nonparametric density estimation , 1996 .

[38]  C. D. Kemp,et al.  Density Estimation for Statistics and Data Analysis , 1987 .

[39]  S. Cummings,et al.  A high ratio of dietary animal to vegetable protein increases the rate of bone loss and the risk of fracture in postmenopausal women 1 – 3 , 2000 .