A Computational Model of Eye Movements during Object Class Detection

We present a computational model of human eye movements in an object class detection task. The model combines state-of-the-art computer vision object class detection methods (SIFT features trained using AdaBoost) with a biologically plausible model of human eye movement to produce a sequence of simulated fixations, culminating with the acquisition of a target. We validated the model by comparing its behavior to the behavior of human observers performing the identical object class detection task (looking for a teddy bear among visually complex non-target objects). We found considerable agreement between the model and human data in multiple eye movement measures, including number of fixations, cumulative probability of fixating the target, and scanpath distance.

[1]  Wei Zhang,et al.  Object class recognition using multiple layer boosting with heterogeneous features , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[2]  Pauline Cockrill,et al.  The Teddy Bear Encyclopedia , 1993 .

[3]  Michael J. Swain,et al.  Color indexing , 1991, International Journal of Computer Vision.

[4]  Rajesh P. N. Rao,et al.  Modeling Saccadic Targeting in Visual Search , 1995, NIPS.

[5]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[6]  Bernhard Schölkopf,et al.  Face Detection - Efficient and Rank Deficient , 2004, NIPS.

[7]  Rajesh P. N. Rao,et al.  PSYCHOLOGICAL SCIENCE Research Article EYE MOVEMENTS REVEAL THE SPATIOTEMPORAL DYNAMICS OE VISUAL SEARCH , 2022 .

[8]  Garrison W. Cottrell,et al.  A model of scan paths applied to face recognition , 2004 .

[9]  David N. Lee,et al.  Where we look when we steer , 1994, Nature.

[10]  Pietro Perona,et al.  Object class recognition by unsupervised scale-invariant learning , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[11]  M. Hayhoe,et al.  In what ways do eye movements contribute to everyday activities? , 2001, Vision Research.

[12]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[13]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[14]  Wilson S. Geisler,et al.  Gaze-contingent real-time simulation of arbitrary visual fields , 2002, IS&T/SPIE Electronic Imaging.

[15]  G. Sperling,et al.  Dynamics of automatic and controlled visual attention. , 1987, Science.

[16]  Wilson S. Geisler,et al.  Real-time foveated multiresolution system for low-bandwidth video communication , 1998, Electronic Imaging.

[17]  F. Keil,et al.  Efficient visual search by category: Specifying the features that mark the difference between artifacts and animals in preattentive vision , 2001, Perception & psychophysics.

[18]  Peter Auer,et al.  Weak Hypotheses and Boosting for Generic Object Detection and Recognition , 2004, ECCV.

[19]  Claudio M. Privitera,et al.  Algorithms for Defining Visual Regions-of-Interest: Comparison with Eye Fixations , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Gregory J. Zelinsky,et al.  Object class recognition using multiple layer boosting with multiple features , 2005, CVPR 2005.

[21]  Rajesh P. N. Rao,et al.  Eye movements in iconic visual search , 2002, Vision Research.

[22]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1995, EuroCOLT.

[23]  Dan Roth,et al.  Learning a Sparse Representation for Object Detection , 2002, ECCV.

[24]  Takeo Kanade,et al.  A statistical method for 3D object detection applied to faces and cars , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[25]  U. Neisser.  VISUAL SEARCH. , 1964, Scientific American.