Is there a safety-net effect with computer-aided detection (CAD)?

Computer-Aided Detection (CAD) systems are used to aid readers interpreting screening mammograms. An expert reader searches the image initially unaided, and then once again with the aid of CAD which prompts automatically detected suspicious regions. This could lead to a ‘safety-net’ effect, where the initial unaided search of the image is adversely affected by the fact that it is preliminary to an additional search with CAD, and may, therefore, be less thorough. To investigate the existence of such an effect, we created a visual search experiment for non-expert observers mirroring breast screening with CAD. Each observer searched 100 images for microcalcification clusters within synthetic images in both prompted and unprompted (no-CAD) conditions. Fifty-two participants were recruited for the study, 48 of whom had their eye movements tracked in real-time; four participants could not be accurately calibrated so only behavioural data was collected. In the CAD condition, before prompts were displayed, image coverage was significantly lower than coverage in the no-CAD condition (t(47)=5.48, p<0.001). Observer sensitivity was significantly greater for targets marked by CAD than the same targets in the no-CAD condition (t(51)=11.67, p<0.001). For targets not marked by CAD, there was no significant difference in observer sensitivity in the CAD condition compared to the same targets in the no-CAD condition (t(51)=0.88, p=0.382). These results suggest that the initial search may be influenced by the subsequent availability of CAD; if so, CAD efficacy studies should account for the effect when estimating benefit.

[1]  Lorenzo Strigini,et al.  Effects of incorrect computer-aided detection (CAD) output on human decision-making in mammography. , 2004, Academic radiology.

[2]  Nico Karssemeijer,et al.  Using computer-aided detection in mammography as a decision support , 2010, European Radiology.

[3]  C. Lehman,et al.  Testing the effect of computer-assisted detection on interpretive performance in screening mammography. , 2006, AJR. American journal of roentgenology.

[4]  John Papaioannou,et al.  Clinically missed cancer: how effectively can radiologists use computer-aided detection? , 2012, AJR. American journal of roentgenology.

[5]  Hilde Bosmans,et al.  Effect of image quality on calcification detection in digital mammography. , 2012, Medical physics.

[6]  Melina A. Kunar,et al.  Low Prevalence Search for Cancers in Mammograms: Evidence Using Laboratory Experiments and Computer Aided Detection , 2017, Journal of experimental psychology. Applied.

[7]  Harold L. Kundel,et al.  Modeling visual search during mammogram viewing , 2004, SPIE Medical Imaging.

[8]  A. Burgess,et al.  Human observer detection experiments with mammograms and power-law noise. , 2001, Medical physics.

[9]  S. Astley,et al.  Computer-aided detection in mammography. , 2004, Clinical radiology.

[10]  J. Elmore,et al.  Variability in interpretive performance at screening mammography and radiologists' characteristics associated with accuracy. , 2009, Radiology.

[11]  David Gur,et al.  The prevalence effect in a laboratory environment: Changing the confidence ratings. , 2007, Academic radiology.

[12]  W Jorritsma,et al.  Improving the radiologist-CAD interaction: designing for appropriate trust. , 2015, Clinical radiology.

[13]  Karla K. Evans,et al.  If You Don’t Find It Often, You Often Don’t Find It: Why Some Cancers Are Missed in Breast Cancer Screening , 2013, PloS one.

[14]  M. Bach.  The Freiburg Visual Acuity test--automatic measurement of visual acuity. , 1996, Optometry and vision science : official publication of the American Academy of Optometry.

[15]  Susan M. Astley,et al.  A citizen science approach to optimising computer aided detection (CAD) in mammography , 2018, Medical Imaging.

[16]  Lorenzo L. Pesce,et al.  Computer-aided detection evaluation methods are not created equal. , 2009, Radiology.

[17]  Melina A. Kunar,et al.  Colour and spatial cueing in low-prevalence visual search , 2012, Quarterly journal of experimental psychology.

[18]  S. Astley,et al.  Single reading with computer-aided detection for screening mammography. , 2008, The New England journal of medicine.

[19]  Sebastiaan Mathôt,et al.  PyGaze: An open-source, cross-platform toolbox for minimal-effort programming of eyetracking experiments , 2014, Behavior research methods.

[20]  Trafton Drew,et al.  When and why might a computer-aided detection (CAD) system interfere with visual search? An eye-tracking study. , 2012, Academic radiology.