Comparing signal-based and case-based methodologies for CAD assessment in a detection task

We are investigating the potential for differences in study conclusions when assessing the estimated impact of a computer-aided detection (CAD) system on readers' performance using a signal-based performance analysis derived from Free-response Receiver Operating Characteristics (FROC) versus a case-based performance analysis derived from Receiver Operating Characteristics (ROC) analysis. To consider this question, we utilized reader data from a CAD assessment study based on 100 mammographic background images to which fixed-size and fixed-intensity Gaussian signals were added, generating a low- and high-intensity set. The study thus allowed CAD assessment in two situations: when CAD sensitivity was 1) superior or 2) equivalent or lower than the average reader. Seven readers were asked to review each set using CAD in both second-reader and concurrent modes. Signal-based detection results were analyzed using the area under the FROC curve below 0.5 false positives per image. Case-based decision results were analyzed using the area under the parametric ROC curve. The results were consistent between the signal-based and case-based analyses for the low-intensity set, suggesting that CAD in both reading modes can increase reader signal-based detection and case-based decision accuracies. For the high-intensity set, the signal-based and case-based analysis suggested different conclusions regarding the utility of CAD, although neither analysis resulted in statistical significance.