Visual Task Inference Using Hidden Markov Models

It has been known for a long time that visual task, such as reading, counting and searching, greatly influences eye movement patterns. Perhaps the best known demonstration of this is the celebrated study of Yarbus showing that different eye movement trajectories emerge depending on the visual task that the viewers are given. The objective of this paper is to develop an inverse Yarbus process whereby we can infer the visual task by observing the measurements of a viewer's eye movements while executing the visual task. The method we are proposing is to use Hidden Markov Models (HMMs) to create a probabilistic framework to infer the viewer's task from eye movements.

[1]  Vinod Nair,et al.  Automated Visual Surveillance Using Hidden Markov Models , 2002 .

[2]  C. Koch,et al.  Task-demands can immediately reverse the effects of sensory-driven saliency in complex visual stimuli. , 2008, Journal of vision.

[3]  Gerhard Tröster,et al.  Eye Movement Analysis for Activity Recognition Using Electrooculography , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[4]  Karl R. Gegenfurtner,et al.  The contribution of low-level features at the centre of gaze to saccade target selection , 2009, Vision Research.

[5]  G. Rizzolatti,et al.  Space and selective attention , 1994 .

[6]  Christof Koch,et al.  Feature combination strategies for saliency-based visual attention systems , 2001, J. Electronic Imaging.

[7]  Benjamin W Tatler,et al.  The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. , 2007, Journal of vision.

[8]  C. Koch,et al.  Probabilistic modeling of eye movement data during conjunction search via feature-based attention. , 2007, Journal of vision.

[9]  J Y Lettvin,et al.  Enhancing the Perception of Form in Peripheral Vision , 1986, Perception.

[10]  Jianying Hu,et al.  HMM Based On-Line Handwriting Recognition , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  F. L. Engel.  Visual conspicuity, directed attention and retinal locus. , 1971, Vision research.

[12]  James J. Clark,et al.  Word ambiguity and the optimal viewing position in reading , 1999, Vision Research.

[13]  John R. Anderson,et al.  Automated Eye-Movement Protocol Analysis , 2001, Hum. Comput. Interact..

[14]  Michael L. Mack,et al.  Viewing task influences eye movement control during active scene perception. , 2009, Journal of vision.

[15]  L. Stark,et al.  Scanpaths in Eye Movements during Pattern Perception , 1971, Science.

[16]  Ronald A. Rensink,et al.  TO SEE OR NOT TO SEE: The Need for Attention to Perceive Changes in Scenes , 1997 .

[17]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[18]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[19]  D. Bouwhuis,et al.  Attention and performance X : control of language processes , 1986 .

[20]  M. Posner,et al.  Components of visual orienting , 1984 .

[21]  A. L. Yarbus.  Eye Movements During Perception of Complex Objects , 1967 .

[22]  Robert M. McPeek,et al.  What neural pathways mediate express saccades? , 1993, Behavioral and Brain Sciences.

[23]  Lawrence W. Stark,et al.  Visual perception and sequences of eye movement fixations: a stochastic modeling approach , 1992, IEEE Trans. Syst. Man Cybern..

[24]  Guang-Zhong Yang,et al.  A gaze-based study for investigating the perception of visual realism in simulated scenes , 2008, TAP.

[25]  C. Koch,et al.  Computational modelling of visual attention , 2001, Nature Reviews Neuroscience.