Prevalence learning and decision making in a visual search task: an equivalent ideal observer approach

Research studies have observed an influence of target prevalence on observer performance for visual search tasks. The goal of this work is to develop models for prevalence effects on visual search. In a recent study by Wolfe et. al, a large scale observer study was conducted to understand the effects of varying target prevalence on visual search. Particularly, a total of 12 observers were recruited to perform 1000 trials of simulated baggage search as target prevalence varied sinusoidally from high to low and back to high. We attempted to model observers’ behavior in prevalence learning and decision making. We modeled the observer as an equivalent ideal observer (EIO) with a prior belief of the signal prevalence. The use of EIO allows the application of ideal observer mathematics to characterize real observers’ performance reading real-life images. For every given new image, the observer updates the belief on prevalence and adjusts his/her decision threshold according to utility theory. The model results agree well with the experimental results from the Wolfe study. The proposed models allow theoretical insights into observer behavior in learning prevalence and adjusting their decision threshold.

[1]  Martin Peterson,et al.  An Introduction to Decision Theory , 2009 .

[2]  C. Beam,et al.  Variability in the interpretation of screening mammograms by US radiologists. Findings from a national sample. , 1996, Archives of internal medicine.

[3]  R. F. Wagner,et al.  Reader Variability in Mammography and Its Implications for Expected Utility over the Population of Readers and Cases , 2004, Medical decision making : an international journal of the Society for Medical Decision Making.

[4]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[5]  J SWETS,et al.  Decision processes in perception. , 1961, Psychological review.

[6]  Brandon D Gallas,et al.  One-shot estimate of MRMC variance: AUC. , 2006, Academic radiology.

[7]  Frank W. Samuelson,et al.  The equivalence of a human observer and an ideal observer in binary diagnostic tasks , 2013, Medical Imaging.

[8]  N Houssami,et al.  Reader variability in reporting breast imaging according to BI-RADS assessment categories (the Florence experience). , 2006, Breast.

[9]  Jeremy M. Wolfe,et al.  Even in correctable search, some types of rare targets are frequently missed , 2009, Attention, perception & psychophysics.

[10]  M P Eckstein,et al.  Visual signal detection in structured backgrounds. I. Effect of number of possible spatial locations and signal contrast. , 1996, Journal of the Optical Society of America. A, Optics, image science, and vision.

[11]  C. Beam,et al.  Effect of human variability on independent double reading in screening mammography. , 1996, Academic radiology.

[12]  Jeremy M Wolfe,et al.  Prevalence of abnormalities influences cytologists' error rates in screening for cervical cancer. , 2011, Archives of pathology & laboratory medicine.

[13]  Karla K. Evans,et al.  If You Don’t Find It Often, You Often Don’t Find It: Why Some Cancers Are Missed in Breast Cancer Screening , 2013, PloS one.

[14]  Rongping Zeng,et al.  Discovering intrinsic properties of human observers' visual search and mathematical observers' scanning. , 2014, Journal of the Optical Society of America. A, Optics, image science, and vision.

[15]  J. Wolfe,et al.  Varying Target Prevalence Reveals Two Dissociable Decision Criteria in Visual Search , 2010, Current Biology.

[16]  Jeremy M Wolfe,et al.  Prevalence effects in newly trained airport checkpoint screeners: trained observers miss rare targets, too. , 2013, Journal of vision.

[17]  J. Swets,et al.  A decision-making theory of visual detection. , 1954, Psychological review.

[18]  David Gur,et al.  Prevalence effect in a laboratory environment. , 2003, Radiology.

[19]  Naomi M. Kenner,et al.  Low target prevalence is a stubborn source of errors in visual search tasks. , 2007, Journal of experimental psychology. General.

[20]  Z. J. Ulehla,et al.  Optimality of perceptual decision criteria. , 1966, Journal of experimental psychology.

[21]  J. P. Thomas,et al.  A signal detection model predicts the effects of set size on visual search accuracy for feature, conjunction, triple conjunction, and disjunction displays , 2000, Perception & psychophysics.

[22]  Shinichi Kita,et al.  The effects of local prevalence and explicit expectations on search termination times , 2012, Attention, perception & psychophysics.

[23]  Bruno de Finetti,et al.  Probabilism , 1989 .