Determination of subjective similarity for pairs of masses and pairs of clustered microcalcifications on mammograms: comparison of similarity ranking scores and absolute similarity ratings.

The presentation of images that are similar to that of an unknown lesion seen on a mammogram may be helpful for radiologists to correctly diagnose that lesion. For similar images to be useful, they must be quite similar from the radiologists' point of view. We have been trying to quantify the radiologists' impression of similarity for pairs of lesions and to establish a "gold standard" for development and evaluation of a computerized scheme for selecting such similar images. However, it is considered difficult to reliably and accurately determine similarity ratings, because they are subjective. In this study, we compared the subjective similarities obtained by two different methods, an absolute rating method and a 2-alternative forced-choice (2AFC) method, to demonstrate that reliable similarity ratings can be determined by the responses of a group of radiologists. The absolute similarity ratings were previously obtained for pairs of masses and pairs of microcalcifications from five and nine radiologists, respectively. In this study, similarity ranking scores for eight pairs of masses and eight pairs of microcalcifications were determined by use of the 2AFC method. In the first session, the eight pairs of masses and eight pairs of microcalcifications were grouped and compared separately for determining the similarity ranking scores. In the second session, another similarity ranking score was determined by use of mixed pairs, i.e., by comparison of the similarity of a mass pair with that of a calcification pair. Four pairs of masses and four pairs of microcalcifications were grouped together to create two sets of eight pairs. The average absolute similarity ratings and the average similarity ranking scores showed very good correlations in the first study (Pearson's correlation coefficients: 0.94 and 0.98 for masses and microcalcifications, respectively). Moreover, in the second study, the correlations between the absolute ratings and the ranking scores were also very high (0.92 and 0.96), which implies that the observers were able to compare the similarity of a mass pair with that of a calcification pair consistently. These results provide evidence that the concept of similarity for pairs of images is robust, even across different lesion types, and that radiologists are able to reliably determine subjective similarity for pairs of breast lesions.

[1]  Ralph L. Day,et al.  Position Bias in Paired Product Tests , 1969 .

[2]  V A Memoli,et al.  Regulation of tumor invasion by cartilage-derived anti-invasion factor in vitro. , 1981, Journal of the National Cancer Institute.

[3]  S Shapiro,et al.  Selection, follow-up, and analysis in the Health Insurance Plan Study: a randomized trial with breast cancer screening. , 1985, National Cancer Institute monograph.

[4]  F. Hall,et al.  Nonpalpable breast lesions: recommendations for biopsy based on suspicion of carcinoma at mammography. , 1988, Radiology.

[5]  A. I. Cohn,et al.  Expert system-controlled image display. , 1989, Radiology.

[6]  L. Tabár,et al.  Update of the Swedish two-county program of mammographic screening for breast cancer. , 1992, Radiologic clinics of North America.

[7]  G J Whitman,et al.  Positive predictive value of breast biopsy performed as a result of mammography: there is no abrupt change at age 50 years. , 1996, Radiology.

[8]  Richard H. Moore,et al.  Current Status of the Digital Database for Screening Mammography , 1998, Digital Mammography / IWDM.

[9]  N. Petrick,et al.  Improvement of radiologists' characterization of mammographic masses by using computer-aided diagnosis: an ROC study. , 1999, Radiology.

[10]  M. Giger,et al.  Improving breast cancer diagnosis with computer-aided diagnosis. , 1999, Academic radiology.

[11]  Eric Y. Tao,et al.  Computer-aided, case-based diagnosis of mammographic regions of interest containing microcalcifications. , 2000, Academic radiology.

[12]  M. Giger,et al.  Breast cancer: effectiveness of computer-aided diagnosis observer study with independent database of mammograms. , 2002, Radiology.

[13]  Robert M. Nishikawa,et al.  Intelligent CAD workstation for breast imaging using similarity to known lesions and multiple visual prompt aids , 2002, SPIE Medical Imaging.

[14]  S. Woolf,et al.  Breast Cancer Screening: A Summary of the Evidence for the U.S. Preventive Services Task Force , 2002, Annals of Internal Medicine.

[15]  K. Doi,et al.  Investigation of new psychophysical measures for evaluation of similar images on thoracic computed tomography for distinction between benign and malignant nodules. , 2003, Medical physics.

[16]  A. Kak,et al.  Automated storage and retrieval of thin-section CT images to assist diagnosis: system description and preliminary assessment. , 2003, Radiology.

[17]  Robert M. Nishikawa,et al.  Observers' ability to judge the similarity of clustered calcifications on mammograms , 2004, SPIE Medical Imaging.

[18]  Nikolas P. Galatsanos,et al.  A similarity learning approach to content-based image retrieval: application to digital mammography , 2004, IEEE Transactions on Medical Imaging.

[19]  Jessica W T Leung,et al.  Performance benchmarks for diagnostic mammography. , 2005, Radiology.

[20]  K. Doi,et al.  Investigation of psychophysical measure for evaluation of similar images for mammographic masses: Preliminary results. , 2005, Medical physics.

[21]  K. Doi,et al.  Investigation of psychophysical measure for evaluation of similar images for mammographic masses: preliminary results. , 2005, Medical physics.

[22]  Li Lan,et al.  Classification of breast lesions with multimodality computer-aided diagnosis: observer study results on an independent clinical data set. , 2006, Radiology.

[23]  K. Doi,et al.  Experimental determination of subjective similarity for pairs of clustered microcalcifications on mammograms: observer study results. , 2006, Medical physics.

[24]  David Gur,et al.  A method to improve visual similarity of breast masses for an interactive computer-aided diagnosis environment. , 2005, Medical physics.

[25]  Paul R. Fisher,et al.  Voice-activated retrieval of mammography reference images , 1998, Journal of Digital Imaging.

[26]  Wesley E. Snyder,et al.  Content-based image retrieval in picture archiving and communications systems , 2009, Journal of Digital Imaging.