Evaluating multiple diagnostic tests with partial verification.

To evaluate diagnostic tests, one would ideally like to verify, for example, with a biopsy, the disease state of all subjects in a study. Often, however, no all subjects are verified. Previous methods for evaluation assume that the decision to verify depends only on recorded variables. Sometimes, particularly if the disease process is not well understood, the decision to verify may also depend on unrecorded variables related to disease. We propose a method to estimate the true- and false-positive rates of multiple tests while adjusting for the effect, on the decision to verify, of unrecorded variables related to disease. To put the estimates into a more usable form, we develop a simple algorithm for creating a receiver-operating curve which maximizes the true-positive rate, given the false-positive rate. We apply the methodology to data on the early detection of prostate cancer using ultrasonography, digital rectal exam, and prostate specific antigen.

[1]  G A Diamond.  Scotched on the ROCs , 1991, Medical decision making : an international journal of the Society for Medical Decision Making.

[2]  P E Shrout,et al.  Design of two-phase prevalence surveys of rare disorders. , 1989, Biometrics.

[3]  Stephen E. Fienberg,et al.  The analysis of cross-classified categorical data , 1980 .

[4]  W F Rosenberger,et al.  Closed-form estimates for missing counts in two-way contingency tables. , 1992, Statistics in medicine.

[5]  C H Brown,et al.  Protecting against nonrandomly missing data in longitudinal studies. , 1990, Biometrics.

[6]  A. Tenenbein.  A Double Sampling Scheme for Estimating from Binomial Data with Misclassifications: Sample size Determination , 1971 .

[7]  S G Baker,et al.  Marginal regression for repeated binary data with outcome subject to non-ignorable non-response. , 1995, Biometrics.

[8]  Juni Palmgren,et al.  The Fisher information matrix for log linear models arguing conditionally on observed explanatory variable , 1981 .

[9]  Donald B. Rubin,et al.  Max-imum Likelihood from Incomplete Data , 1972 .

[10]  Stuart G. Baker,et al.  Evaluating a new test using a reference test with estimated sensitivity and specificity , 1991 .

[11]  Robert N. Rodriguez,et al.  Statistical effects of imperfect inspection sampling: II. Double sampling and link sampling , 1985 .

[12]  Conditional resampling for misclassified multinomial data with applications to sampling inspection , 1992 .

[13]  Stuart G. Baker,et al.  A Simple Method for Computing the Observed Information Matrix When Using the EM Algorithm with Categorical Data , 1992 .

[14]  A. Tenenbein.  A Double Sampling Scheme for Estimating from Misclassified Multinomial Data with Applications to Sampling Inspection , 1972 .

[15]  J. Knottnerus.  The Effects of Disease Verification and Referral on the Relationship Between Symptoms and Diseases , 1987, Medical decision making : an international journal of the Society for Medical Decision Making.

[16]  S G Baker,et al.  Regression analysis of grouped survival data: informative censoring and double sampling. , 1993, Biometrics.

[17]  D. Heisey,et al.  Use of a Log‐Linear Model with the EM Algorithm to Correct Estimates of Stock Composition and to Convert Length to Age , 1987 .

[18]  George A. Diamond,et al.  ROC Steady , 1987, Medical decision making : an international journal of the Society for Medical Decision Making.

[19]  W. Cooner,et al.  Prostate cancer detection in a clinical urological practice by ultrasonography, digital rectal examination and prostate specific antigen. , 1990, The Journal of urology.

[20]  Jesse C. Arnold,et al.  A Two-Sample Test for Independence in 2×2 Contingency Tables with Both Margins Subject to Misclassification , 1977 .

[21]  Charles L. Odoroff,et al.  Log-Linear Models for Doubly Sampled Categorical Data Fitted by the EM Algorithm , 1985 .

[22]  Nan M. Laird,et al.  Regression Analysis for Categorical Variables with Outcome Subject to Nonignorable Nonresponse , 1988 .

[23]  T. Chen,et al.  Log-Linear Models for Categorical Data with Misclassification and Double Sampling , 1979 .

[24]  Yosef Hochberg,et al.  On the Use of Double Sampling Schemes in Analyzing Categorical Data with Misclassification Errors , 1977 .

[25]  R A Greenes,et al.  Assessment of diagnostic tests when disease verification is subject to selection bias. , 1983, Biometrics.

[26]  Xiao-Hua Zhou,et al.  Maximum likelihood estimators of sensitivity and specificity corrected for verification bias , 1993 .

[27]  Diamond's Correction Method-A Real Gem or Just Cubic Zirconium? , 1991, Medical decision making : an international journal of the Society for Medical Decision Making.

[28]  Marvin Zelen,et al.  Testing hypotheses with binary data subject to misclassification errors: Analysis and experimental design , 1991 .

[29]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[30]  G. Diamond.  Reverend Bayes' silent majority. An alternative factor affecting sensitivity and specificity of exercise electrocardiography. , 1986, The American journal of cardiology.

[31]  X H Zhou,et al.  Effect of verification bias on positive and negative predictive values. , 1994, Statistics in medicine.

[32]  C B Begg,et al.  Testing for Fetal Pulmonary Maturity , 1990, Medical decision making : an international journal of the Society for Medical Decision Making.

[33]  R A Greenes,et al.  Construction of Receiver Operating Characteristic Curves when Disease Verification Is Subject to Selection Bias , 1984, Medical decision making : an international journal of the Society for Medical Decision Making.

[34]  M. Weinstein,et al.  Clinical Decision Analysis , 1980 .

[35]  Juni Palmgren,et al.  Precision of double sampling estimators for comparing two probabilities , 1987 .