Decision strategies that maximize the area under the LROC curve

For the 2-class detection problem (signal absent/present), the likelihood ratio is an ideal observer in that it minimizes Bayes risk for arbitrary costs and it maximizes the area under the receiver operating characteristic (ROC) curve [AUC]. The AUC-optimizing property makes it a valuable tool in imaging system optimization. If one considered a different task, namely, joint detection and localization of the signal, then it would be similarly valuable to have a decision strategy that optimized a relevant scalar figure of merit. We are interested in quantifying performance on decision tasks involving location uncertainty using the localization ROC (LROC) methodology. Therefore, we derive decision strategies that maximize the area under the LROC curve, A/sub LROC/. We show that these decision strategies minimize Bayes risk under certain reasonable cost constraints. The detection-localization task is modeled as a decision problem in three increasingly realistic ways. In the first two models, we treat location as a discrete parameter having finitely many values resulting in an (L+1) class classification problem. In our first simple model, we do not include search tolerance effects and in the second, more general, model, we do. In the third and most general model, we treat location as a continuous parameter and also include search tolerance effects. In all cases, the essential proof that the observer maximizes A/sub LROC/ is obtained with a modified version of the Neyman-Pearson lemma. A separate form of proof is used to show that in all three cases, the decision strategy minimizes the Bayes risk under certain reasonable cost constraints.

[1]  Matthew A. Kupinski,et al.  Ideal-Observer Performance under Signal and Background Uncertainty , 2003, IPMI.

[2]  Craig K. Abbey,et al.  Detection performance theory for ultrasound imaging systems , 2005, IEEE Transactions on Medical Imaging.

[3]  Shigeo Abe DrEng Pattern Classification , 2001, Springer London.

[4]  Berkman Sahiner,et al.  Design of three-class classifiers in computer-aided diagnosis: Monte Carlo simulation study , 2003, SPIE Medical Imaging.

[5]  J Llacer,et al.  ROC and LROC analyses of the effects of lesion contrast, size, and signal-to-noise ratio on detectability in PET images. , 2000, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.

[6]  Matthew A. Kupinski,et al.  Objective Assessment of Image Quality , 2005 .

[7]  Craig K. Abbey,et al.  Bayesian Detection of Random Signals on Random Backgrounds , 1997, IPMI.

[8]  Eric Clarkson,et al.  Efficiency of the human observer detecting random signals in random backgrounds. , 2005, Journal of the Optical Society of America. A, Optics, image science, and vision.

[9]  H. Barrett,et al.  Objective assessment of image quality. III. ROC metrics, ideal observers, and likelihood-generating functions. , 1998, Journal of the Optical Society of America. A, Optics, image science, and vision.

[10]  Steven Kay,et al.  Fundamentals Of Statistical Signal Processing , 2001 .

[11]  Gerry Leversha,et al.  Statistical inference (2nd edn), by Paul H. Garthwaite, Ian T. Jolliffe and Byron Jones. Pp.328. £40 (hbk). 2002. ISBN 0 19 857226 3 (Oxford University Press). , 2003, The Mathematical Gazette.

[12]  R. F. Wagner,et al.  Unified SNR analysis of medical imaging systems , 1985, Physics in medicine and biology.

[13]  Robert M. Nishikawa,et al.  Hypervolume under the ROC hypersurface of a near-guessing ideal observer in a three-class classification task , 2004, SPIE Medical Imaging.

[14]  Jinyi Qi,et al.  Fast approach to evaluate MAP reconstruction for lesion detection and localization , 2004, SPIE Medical Imaging.

[15]  Robert M. Nishikawa,et al.  The hypervolume under the ROC hypersurface of "Near-Guessing" and "Near-Perfect" observers in N-class classification tasks , 2005, IEEE Transactions on Medical Imaging.

[16]  D G Brown,et al.  Detection performance of the ideal decision function and its McLaurin expansion: signal position unknown. , 1995, The Journal of the Acoustical Society of America.

[17]  Gene Gindi,et al.  Fast LROC analysis of Bayesian reconstructed emission tomographic images using model observers. , 2005, Physics in medicine and biology.

[18]  Michael A. King,et al.  LROC analysis of detector-response compensation in SPECT , 2000, IEEE Transactions on Medical Imaging.

[19]  P. Khurd,et al.  Rapid computation of LROC figures of merit using numerical observers (for SPECT/PET reconstruction) , 2005, IEEE Transactions on Nuclear Science.

[20]  Eric C Frey,et al.  Assessment of scatter compensation strategies for (67)Ga SPECT using numerical observers and human LROC studies. , 2004, Journal of nuclear medicine : official publication, Society of Nuclear Medicine.

[21]  Kyle J. Myers,et al.  Foundations of Image Science , 2003, J. Electronic Imaging.

[22]  David Jaarsma,et al.  More on the Detection of One of M Orthogonal Signals , 1967 .

[23]  David G. Stork,et al.  Pattern Classification , 1973 .

[24]  Matthew A. Kupinski,et al.  Ideal observers and optimal ROC hypersurfaces in N-class classification , 2004, IEEE Transactions on Medical Imaging.

[25]  Matthew A. Kupinski,et al.  A fast model of a multiple-pinhole SPECT imaging system , 2005, SPIE Medical Imaging.

[26]  G. Casella,et al.  Statistical Inference , 2003, Encyclopedia of Social Network Analysis and Mining.

[27]  R. Swensson Unified measurement of observer performance in detecting and localizing target objects on images. , 1996, Medical physics.

[28]  H. Barrett,et al.  Ideal-observer computation in medical imaging with use of Markov-chain Monte Carlo techniques. , 2003, Journal of the Optical Society of America. A, Optics, image science, and vision.

[29]  Keinosuke Fukunaga,et al.  Statistical Pattern Recognition , 1993, Handbook of Pattern Recognition and Computer Vision.

[30]  Michael A. King,et al.  Assessment of scatter compensation strategies for /sup 67/Ga tumor SPECT using numerical observers and human LROC studies , 2002, 2002 IEEE Nuclear Science Symposium Conference Record.

[31]  Maria Kallergi,et al.  Improved interpretation of digitized mammography with wavelet processing: a localization response operating characteristic study. , 2004, AJR. American journal of roentgenology.