Recent developments in imaging system assessment methodology, FROC analysis and the search model.

A frequent problem in imaging is assessing whether a new imaging system is an improvement over an existing standard. Observer performance methods, in particular the receiver operating characteristic (ROC) paradigm, are widely used in this context. In ROC analysis lesion location information is not used and consequently scoring ambiguities can arise in tasks, such as nodule detection, involving finding localized lesions. This paper reviews progress in the free-response ROC (FROC) paradigm in which the observer marks and rates suspicious regions and the location information is used to determine whether lesions were correctly localized. Reviewed are FROC data analysis, a search-model for simulating FROC data, predictions of the model and a method for estimating the parameters. The search model parameters are physically meaningful quantities that can guide system optimization.

[1]  R. Swensson.  Unified measurement of observer performance in detecting and localizing target objects on images. , 1996, Medical physics.

[2]  Claudia Mello-Thoms,et al.  Spatial localization accuracy of radiologists in free-response studies: Inferring perceptual FROC curves from mark-rating data. , 2007, Academic radiology.

[3]  Dev P Chakraborty,et al.  Observer studies involving detection and localization: modeling, analysis, and validation. , 2004, Medical physics.

[4]  C E Metz,et al.  Some practical issues of experimental design and data analysis in radiological ROC studies. , 1989, Investigative radiology.

[5]  K. Berbaum,et al.  Receiver operating characteristic rating analysis. Generalization to the population of readers and patients with the jackknife method. , 1992, Investigative radiology.

[6]  C. Metz.  ROC Methodology in Radiologic Imaging , 1986, Investigative radiology.

[7]  N A Obuchowski,et al.  Data analysis for detection and localization of multiple abnormalities with application to mammography. , 2000, Academic radiology.

[8]  Stephen L Hillis,et al.  Recent developments in the Dorfman-Berbaum-Metz procedure for multireader ROC study analysis. , 2008, Academic radiology.

[9]  H. Miller.  The FROC curve: a representation of the observer's performance for the method of free response. , 1969, The Journal of the Acoustical Society of America.

[10]  C E Metz,et al.  The "proper" binormal model: parametric receiver operating characteristic curve estimation with degenerate data. , 1997, Academic radiology.

[11]  D. Chakraborty.  ROC curves predicted by a model of visual search , 2006, Physics in medicine and biology.

[12]  Lorenzo L. Pesce,et al.  Reliable and computationally efficient maximum-likelihood estimation of "proper" binormal ROC curves. , 2007, Academic radiology.

[13]  D P Chakraborty.  A search model and figure of merit for observer data acquired according to the free-response paradigm. , 2006, Physics in medicine and biology.

[14]  James P. Egan,et al.  Operating Characteristics, Signal Detectability, and the Method of Free Response , 1961 .

[15]  John F. Hamilton,et al.  A Free Response Approach To The Measurement And Characterization Of Radiographic Observer Performance , 1977, Other Conferences.

[16]  Ce. Metz,et al.  Receiver operating characteristic (ROC) analysis in medical imaging , 1997 .

[17]  C. Metz,et al.  Visual detection and localization of radiographic images. , 1975, Radiology.

[18]  Dev P Chakraborty,et al.  Validation and statistical power comparison of methods for analyzing free-response observer performance studies. , 2008, Academic radiology.

[19]  C. Metz.  Basic principles of ROC analysis. , 1978, Seminars in nuclear medicine.

[20]  Hong-Jun Yoon,et al.  Operating characteristics predicted by models for diagnostic tasks involving lesion localization. , 2008, Medical physics.

[21]  Ehsan Samei,et al.  The Handbook of Medical Image Perception and Techniques , 2009 .

[22]  D. Dorfman,et al.  Maximum-likelihood estimation of parameters of signal-detection theory and determination of confidence intervals—Rating-method data , 1969 .