Metrological characterization of a diagnostic test extending the Receiving Operating Curve analysis using Supplement 2 recommendations

Receiving Operating Curve (ROC) analysis is a powerful and statistical accepted method to assess the performance of a diagnostic test. ROC curve plots true positive rate against false positive rate, evaluated on a certain population. Instrumental and model uncertainty contributions can strongly affect the performance of the ROC analysis especially in the evaluation of performance metrics such as Area Under ROC (AUC) and Optimal Operating Points. Supplement 2 reports detailed instructions to handle and propagate uncertainty through a Multiple Input Multiple Output system, in case of correlate output variables, such as TPR and FPR. After a detailed revision of the existing literature, the present paper describes and applies a novel methodology, totally framed in the GUM and its supplements, to represent and propagate the uncertainty contributions estimated in a medical context, throughout the ROC analysis, providing new concepts such as ROC confidence region and Optimal Operating Region.

[1]  B. Taylor,et al.  Status Report on the ISO/IEC/OIML/BIPM Guide, Expression of Uncertainty in Measurement , 1992 .

[2]  A James O'Malley,et al.  A Bayesian Hierarchical Non‐Linear Regression Model in Receiver Operating Characteristic Analysis of Clustered Continuous Diagnostic Data , 2005, Biometrical journal. Biometrische Zeitschrift.

[3]  S. Standard.  GUIDE TO THE EXPRESSION OF UNCERTAINTY IN MEASUREMENT , 2006 .

[4]  G. Campbell,et al.  Advances in statistical methodology for the evaluation of diagnostic and laboratory tests. , 1994, Statistics in medicine.

[5]  N. Obuchowski.  Receiver operating characteristic curves and their use in radiology. , 2003, Radiology.

[6]  Ian W. Ricketts,et al.  The Mammographic Image Analysis Society digital mammogram database , 1994 .

[7]  M. Zweig,et al.  Receiver-operating characteristic (ROC) plots: a fundamental evaluation tool in clinical medicine. , 1993, Clinical chemistry.

[8]  Gerda Claeskens,et al.  Empirical likelihood confidence regions for comparison distributions and roc curves , 2003 .

[9]  R. Hilgers.  Distribution-Free Confidence Bounds for ROC Curves , 1991, Methods of Information in Medicine.

[10]  H. Schäfer,et al.  Efficient confidence bounds for ROC curves. , 1994, Statistics in medicine.

[11]  M. Salmeri,et al.  Bilateral asymmetry identification for the early detection of breast cancer , 2011, 2011 IEEE International Symposium on Medical Measurements and Applications.

[12]  W. Hall,et al.  Confidence Bands for Receiver Operating Characteristic Curves , 1993, Medical decision making : an international journal of the Society for Medical Decision Making.

[13]  John Eng,et al.  Receiver operating characteristic analysis: a primer. , 2005, Academic radiology.

[14]  M. Pepe.  The Statistical Evaluation of Medical Tests for Classification and Prediction , 2003 .

[15]  L. Liberman,et al.  Breast imaging reporting and data system (BI-RADS). , 2002, Radiologic clinics of North America.

[16]  C. Metz,et al.  Maximum likelihood estimation of receiver operating characteristic (ROC) curves from continuously-distributed data. , 1998, Statistics in medicine.

[17]  A. Mencattini,et al.  Signal-dependent noise characterization for mammographic images denoising , 2008 .

[18]  Rangaraj M. Rangayyan,et al.  Recent Advances in Breast Imaging, Mammography, and Computer-Aided Diagnosis of Breast Cancer , 2006 .

[19]  J A Hanley,et al.  Bootstrap confidence intervals for the sensitivity of a quantitative diagnostic test. , 2000, Statistics in medicine.

[20]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[21]  Arianna Mencattini,et al.  Noise estimation in mammographic images for adaptive denoising , 2007 .

[22]  K Linnet,et al.  Comparison of quantitative diagnostic tests: type I error, power, and sample size. , 1987, Statistics in medicine.

[23]  Arianna Mencattini,et al.  Metrological Assessment of a CAD System for the Early Diagnosis of Breast Cancer in Digital Mammography , 2012 .

[24]  D. Parker,et al.  Uncertainty Estimation for Target Detection System Discrimination and Confidence Performance Metrics , 2006 .

[25]  Mark E. Oxley,et al.  Development of a Bayesian Framework for Determining Uncertainty in Receiver Operating Characteristic Curve Estimates , 2010, IEEE Transactions on Knowledge and Data Engineering.

[26]  K. Zou,et al.  Smooth non-parametric receiver operating characteristic (ROC) curves for continuous diagnostic tests. , 1997, Statistics in medicine.