Optimal visual search strategy with inter-saccade response correlations

Humans process the visual world with varying resolution across the visual field and sample information using eye movements to point the high-resolution central vision (fovea) to regions of interest. What eye movements optimize decisions in tasks such as target detection and localization? In 2005, Najemnik & Geisler proposed the Bayesian Ideal searcher (IS) that takes into account the foveated properties of human visual and employs the optimal fixation selection strategy to maximize visual search performance. In addition, they proposed a computationally simpler model, entropy limit minimization (ELM), that approximates the IS searcher. One limitation of these models is that they were developed with the assumption that the visual system’s internal responses across fixations are statistically independent. This assumption will not hold for search tasks such as in 2D medical images for which the external noise and anatomical noise results in correlations in internal responses across saccadic fixations (inter-saccade response correlations). In this work, we present image-computable foveated IS and ELM models that accommodate inter-saccade response correlations. We demonstrate that for static images, the optimal searchers that account for the inter-saccade correlations (i.e., IS-COR and ELM-COR) significantly outperform the traditional methods that ignore such correlations. Moreover, the novel ELM-COR achieves a similar performance as the IS-COR but runs about 18 times faster. Together, the IS-COR and ELM-COR extend the optimal searcher framework to evaulate human fixations for more realistic search tasks with inter-saccadic correlations.

[1]  M. Eckstein,et al.  A deep Q-learning method for optimizing visual search strategies in backgrounds of dynamic noise , 2022, Medical Imaging.

[2]  Mark A. Anastasio,et al.  A Hybrid Approach for Approximating the Ideal Observer for Joint Signal Detection and Estimation Tasks by Use of Supervised Learning and Markov-Chain Monte Carlo Methods , 2021, IEEE Transactions on Medical Imaging.

[3]  F. Brooks,et al.  Learning stochastic object models from medical imaging measurements by use of advanced ambient generative adversarial networks , 2021, Journal of medical imaging.

[4]  Andrew D. A. Maidment,et al.  Under-exploration of Three-Dimensional Images Leads to Search Errors for Small Salient Targets , 2021, Current Biology.

[5]  Craig K. Abbey,et al.  Foveated Model Observers for Visual Search in 3D Medical Images , 2020, IEEE Transactions on Medical Imaging.

[6]  Mark A. Anastasio,et al.  Approximating the Ideal Observer for Joint Signal Detection and Localization Tasks by use of Supervised Learning Methods , 2020, IEEE Transactions on Medical Imaging.

[7]  Craig K. Abbey,et al.  Foveated model observer to predict human search performance on virtual digital breast tomosynthesis phantoms , 2020, Medical Imaging.

[8]  Mark A. Anastasio,et al.  Learning numerical observers using unsupervised domain adaptation , 2020, Medical Imaging.

[9]  Mark A. Anastasio,et al.  Markov-Chain Monte Carlo approximation of the Ideal Observer using generative adversarial networks , 2020, Medical Imaging.

[10]  Hua Li,et al.  Approximating the Ideal Observer and Hotelling Observer for Binary Signal Detection Tasks by Use of Supervised Learning Methods , 2019, IEEE Transactions on Medical Imaging.

[11]  Sheng Zhang,et al.  Optimal and human eye movements to clustered low value cues to increase decision rewards during search , 2015, Vision Research.

[12]  Michael S Landy,et al.  Statistical templates for visual search. , 2014, Journal of vision.

[13]  Trafton Drew,et al.  Scanners and drillers: characterizing expert visual search through volumetric images. , 2013, Journal of vision.

[14]  Raymond Bertram,et al.  The Effect of Expertise on Eye Movement Behaviour in Medical Image Perception , 2013, PloS one.

[15]  Miguel P. Eckstein,et al.  Evolution and Optimality of Similar Neural Mechanisms for Perception and Action during Search , 2010, PLoS Comput. Biol..

[16]  Wilson S. Geisler,et al.  Simple summation rule for optimal fixation selection in visual search , 2009, Vision Research.

[17]  T. Crawford,et al.  How do radiologists do it? The influence of experience and training on searching for chest nodules. , 2006 .

[18]  Wilson S. Geisler,et al.  Optimal eye movement strategies in visual search , 2005, Nature.

[19]  Harold L. Kundel,et al.  Reader error, object recognition, and visual search , 2004, SPIE Medical Imaging.

[20]  Miguel P. Eckstein,et al.  Evaluation of JPEG 2000 encoder options: human and model observer detection of variable signals in X-ray coronary angiograms , 2004, IEEE Transactions on Medical Imaging.

[21]  Craig K. Abbey,et al.  Estimation of human-observer templates in two-alternative forced-choice experiments , 1999, Medical Imaging.

[22]  E. Krupinski,et al.  Searching for lung nodules. Visual dwell indicates locations of false-positive and false-negative decisions. , 1989, Investigative radiology.

[23]  Andrew B. Watson,et al.  Detection and Recognition of Simple Spatial Forms , 1983 .