Head and eye gaze dynamics during visual attention shifts in complex environments.

The dynamics of overt visual attention shifts evoke certain patterns of responses in eye and head movements. In this work, we detail novel findings regarding the interaction of eye gaze and head pose under various attention-switching conditions in complex environments and safety critical tasks such as driving. In particular, we find that sudden, bottom-up visual cues in the periphery evoke a different pattern of eye-head movement latencies as opposed to those during top-down, task-oriented attention shifts. In laboratory vehicle simulator experiments, a unique and significant (p < 0.05) pattern of preparatory head motions, prior to the gaze saccade, emerges in the top-down case. This finding is validated in qualitative analysis of naturalistic real-world driving data. These results demonstrate that measurements of eye-head dynamics are useful data for detecting driver distractions, as well as in classifying human attentive states in time and safety critical tasks.

[1]  H. Gray Anatomy, Descriptive and Surgical , 1858, Glasgow Medical Journal.

[2]  R. Dodge,et al.  The Latent Time of Compensatory Eye-movements. , 1921 .

[3]  A. Bartz Eye and Head Movements in Peripheral Vision: Nature of Compensatory Eye Movements , 1966, Science.

[4]  A. Fuchs,et al.  Activity of brain stem neurons during eye movements of alert monkeys. , 1972, Journal of neurophysiology.

[5]  E Bizzi,et al.  Two modes of active eye-head coordination in monkeys. , 1972, Brain research.

[6]  R. Wurtz,et al.  Activity of superior colliculus in behaving monkey. IV. Effects of lesions on eye movements. , 1972, Journal of neurophysiology.

[7]  E. Keller Participation of medial pontine reticular formation in eye movement generation in monkey. , 1974, Journal of neurophysiology.

[8]  Giulio Sandini,et al.  Control Strategies in the Eye-Head Coordination System , 1977, IEEE Transactions on Systems, Man, and Cybernetics.

[9]  R H Wurtz,et al.  Role of striate cortex and superior colliculus in visual guidance of saccadic eye movements in monkeys. , 1977, Journal of neurophysiology.

[10]  J. L. Conway,et al.  Deficits in eye movements following frontal eye-field and superior colliculus ablations. , 1980, Journal of neurophysiology.

[11]  M. Posner,et al.  Orienting of Attention* , 1980, The Quarterly journal of experimental psychology.

[12]  L. Stark,et al.  Types of gaze movement: Variable interactions of eye and head movements , 1982, Experimental Neurology.

[13]  M. Posner,et al.  Neural systems control of spatial orienting. , 1982, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[14]  R H Wurtz,et al.  Deficits in eye position following ablation of monkey superior colliculus, pretectum, and posterior-medial thalamus. , 1982, Journal of neurophysiology.

[15]  M. Posner,et al.  Components of visual orienting , 1984 .

[16]  M. Posner,et al.  Inhibition of return : Neural basis and function , 1985 .

[17]  E. Maylor,et al.  Inhibitory component of externally controlled covert orienting in visual space. , 1985, Journal of experimental psychology. Human perception and performance.

[18]  J. van der Steen,et al.  Effects of unilateral frontal eye-field lesions on eye-head coordination in monkey. , 1986, Journal of neurophysiology.

[19]  D. Guitton,et al.  Control of eye—head coordination during orienting gaze shifts , 1992, Trends in Neurosciences.

[20]  J. Fuller,et al.  Comparison of Head Movement Strategies among Mammals , 1992 .

[21]  Philippe Lefèvre,et al.  Dynamic feedback to the superior colliculus in a neural network model of the gaze control system , 1992, Neural Networks.

[22]  Michael F. Land,et al.  Predictable eye-head coordination during driving , 1992, Nature.

[23]  D Guitton,et al.  Central Organization and Modeling of Eye‐Head Coordination during Orienting Gaze Shifts a , 1992, Annals of the New York Academy of Sciences.

[24]  A Berthoz,et al.  Saccade‐vestibulo‐ocular reflex co‐operation and eye‐head uncoupling during orientation to flashed target. , 1993, The Journal of physiology.

[25]  R. Abrams,et al.  Inhibition of return: effects of attentional cuing on eye movement latencies. , 1994, Journal of experimental psychology. Human perception and performance.

[26]  R. Rafal,et al.  Effects of inhibition of return on voluntary and visually guided saccades. , 1994, Canadian journal of experimental psychology = Revue canadienne de psychologie experimentale.

[27]  J. Hoffman,et al.  The role of visual attention in saccadic eye movements , 1995, Perception & psychophysics.

[28]  B. Dosher,et al.  The role of attention in the programming of saccades , 1995, Vision Research.

[29]  S. Shimojo,et al.  Location vs Feature: Reaction Time Reveals Dissociation Between Two Visual Functions , 1996, Vision Research.

[30]  Michael C. Mozer,et al.  Computational modeling of spatial attention , 1996 .

[31]  D. Sparks,et al.  Combined eye-head gaze shifts produced by electrical stimulation of the superior colliculus in rhesus monkeys. , 1996, Journal of neurophysiology.

[32]  H. Deubel,et al.  Saccade target selection and object recognition: Evidence for a common attentional mechanism , 1996, Vision Research.

[33]  Amishi P. Jha,et al.  What Is Inhibited in Inhibition of Return , 1996 .

[34]  A. Opstal,et al.  Human eye-head coordination in two dimensions under different sensorimotor conditions , 1997, Experimental Brain Research.

[35]  D L Sparks,et al.  Activity of cells in the deeper layers of the superior colliculus of the rhesus monkey: evidence for a gaze displacement command. , 1997, Journal of neurophysiology.

[36]  D. Sparks,et al.  Eye-head coordination during head-unrestrained gaze shifts in rhesus monkeys. , 1997, Journal of neurophysiology.

[37]  R. Klein,et al.  Inhibition of return to color: A replication and nonextension of Law, Pratt, and Abrams (1995) , 1998, Perception & psychophysics.

[38]  M. Corbetta,et al.  A Common Network of Functional Areas for Attention and Eye Movements , 1998, Neuron.

[39]  Robert M. McPeek,et al.  Saccades require focal attention and are facilitated by a short-term memory system , 1999, Vision Research.

[40]  Kim M Cardosi,et al.  HUMAN FACTORS FOR AIR TRAFFIC CONTROL SPECIALISTS: A USER'S MANUAL FOR YOUR BRAIN. , 1999 .

[41]  D. Munoz,et al.  Human eye-head gaze shifts in a distractor task. I. Truncated gaze shifts. , 1999, Journal of neurophysiology.

[42]  Alex Pentland,et al.  Modeling and Prediction of Human Behavior , 1999, Neural Computation.

[43]  D P Munoz,et al.  Human eye-head gaze shifts in a distractor task. II. Reduced threshold for initiation of early head movements. , 1999, Journal of neurophysiology.

[44]  Alex Pentland,et al.  A Bayesian Computer Vision System for Modeling Human Interactions , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  M. A. Recarte,et al.  Effects of verbal and spatial-imagery tasks on eye fixations while driving. , 2000, Journal of experimental psychology. Applied.

[46]  Alexander Zelinsky,et al.  Behavior recognition based on head pose and gaze direction measurement , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[47]  C. Koch,et al.  A saliency-based search mechanism for overt and covert shifts of visual attention , 2000, Vision Research.

[48]  E G Keating,et al.  Electrical stimulation of the frontal eye field in a monkey produces combined eye and head movements. , 2000, Journal of neurophysiology.

[49]  Leslie G. Ungerleider,et al.  Mechanisms of visual attention in the human cortex. , 2000, Annual review of neuroscience.

[50]  R. Klein,et al.  Inhibition of return , 2000, Trends in Cognitive Sciences.

[51]  M. Hayhoe,et al.  The coordination of eye, head, and hand movements in a natural task , 2001, Experimental Brain Research.

[52]  R. Steinman,et al.  Temporal coordination of the human head and eye during a natural sequential tapping task , 2001, Vision Research.

[53]  Michael S. Beauchamp,et al.  A Parametric fMRI Study of Overt and Covert Shifts of Visuospatial Attention , 2001, NeuroImage.

[54]  T Moore,et al.  Control of eye movements and spatial attention. , 2001, Proceedings of the National Academy of Sciences of the United States of America.

[55]  J. C. Johnston,et al.  Attention and performance. , 2001, Annual review of psychology.

[56]  N. J. Gandhi,et al.  Cortical and subcortical contributions to coordinated eye and head movements , 2001, Vision Research.

[57]  Steven J. Landry,et al.  A methodology for studying cognitive groupings in a target-tracking task , 2001, IEEE Trans. Intell. Transp. Syst..

[58]  S de Brouwer,et al.  Role of retinal slip in the prediction of target motion during smooth and saccadic pursuit. , 2001, Journal of neurophysiology.

[59]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[60]  Eliana M. Klier,et al.  The superior colliculus encodes gaze commands in retinal coordinates , 2001, Nature Neuroscience.

[61]  M. Missal,et al.  Quantitative analysis of catch-up saccades during sustained pursuit. , 2002, Journal of neurophysiology.

[62]  R. Klein,et al.  Contribution of the Primate Superior Colliculus to Inhibition of Return , 2002, Journal of Cognitive Neuroscience.

[63]  J. Stahl,et al.  The influence of future gaze orientation upon eye-head coupling during saccades , 2004, Experimental Brain Research.

[64]  Hongying Wang,et al.  Contribution of head movement to gaze command coding in monkey frontal cortex and superior colliculus. , 2003, Journal of neurophysiology.

[65]  Daniel Guitton,et al.  On the feedback control of orienting gaze shifts made with eye and head movements. , 2003, Progress in brain research.

[66]  Michael E. Tipping,et al.  Fast Marginal Likelihood Maximisation for Sparse Bayesian Models , 2003 .

[67]  Etienne Olivier,et al.  Visual Responses on Neck Muscles Reveal Selective Gating that Prevents Express Saccades , 2004, Neuron.

[68]  Paula J. Durlach,et al.  Change Blindness and Its Implications for Complex Monitoring and Control Systems Design and Operator Training , 2004, Hum. Comput. Interact..

[69]  M Cameron,et al.  World Report on Road Traffic Injury Prevention. , 2004 .

[70]  Suzanne E. Lee,et al.  A COMPREHENSIVE EXAMINATION OF NATURALISTIC LANE-CHANGES , 2004 .

[71]  C. Pierrot-Deseilligny,et al.  Eye movement control by the cerebral cortex , 2004, Current opinion in neurology.

[72]  Lance M. Optican,et al.  Recovery of saccadic dysmetria following localized lesions in monkey superior colliculus , 2004, Experimental Brain Research.

[73]  Robert H. Wurtz,et al.  Subcortical Modulation of Attention Counters Change Blindness , 2004, The Journal of Neuroscience.

[74]  Avi Caspi,et al.  The decoupling of attention and eye movements during multiple fixation search , 2004 .

[75]  Joyce Vliegen,et al.  Dynamic Sound Localization during Rapid Eye-Head Gaze Shifts , 2004, The Journal of Neuroscience.

[76]  Peter W Dicke,et al.  Neuron-specific contribution of the superior colliculus to overt and covert shifts of attention , 2004, Nature Neuroscience.

[77]  Julio C. Martinez-Trujillo,et al.  Frames of Reference for Eye-Head Gaze Commands in Primate Supplementary Eye Fields , 2004, Neuron.

[78]  R. Krauzlis Recasting the smooth pursuit eye movement system. , 2004, Journal of neurophysiology.

[79]  B. Fischer,et al.  Separate populations of visually guided saccades in humans: reaction times and amplitudes , 2004, Experimental Brain Research.

[80]  Margaret M. Peden,et al.  World Report on Road Traffic Injury Prevention , 2004 .

[81]  A John Van Opstal,et al.  Gaze orienting in dynamic visual double steps. , 2005, Journal of neurophysiology.

[82]  James R Müller,et al.  Microstimulation of the superior colliculus focuses attention without moving the eyes. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[83]  Mark M G Walton,et al.  Head movement evoked by electrical stimulation in the supplementary eye field of the rhesus monkey. , 2005, Journal of neurophysiology.

[84]  L. Itti,et al.  Modeling the influence of task on attention , 2005, Vision Research.

[85]  Takashi R Sato,et al.  Neuronal Basis of Covert Spatial Attention in the Frontal Eye Field , 2005, The Journal of Neuroscience.

[86]  A Rawat,et al.  World Report on Road Traffic Injury Prevention: a critique , 2005 .

[87]  Wei Zhang,et al.  The Role of Top-down and Bottom-up Processes in Guiding Eye Movements during Visual Search , 2005, NIPS.

[88]  Brian D Corneil,et al.  Countermanding eye-head gaze shifts in humans: marching orders are delivered to the head first. , 2005, Journal of neurophysiology.

[89]  R. Krauzlis The Control of Voluntary Eye Movements: New Perspectives , 2005, The Neuroscientist : a review journal bringing neurobiology, neurology and psychiatry.

[90]  J. Stahl,et al.  Amplitudes of head movements during putative eye-only saccades , 2005, Brain Research.

[91]  Jorge Batista,et al.  A Real-Time Driver Visual Attention Monitoring System , 2005, IbPRIA.

[92]  Hongying Wang,et al.  Cross-coupled adaptation of eye and head position commands in the primate gaze control system , 2005, Neuroreport.

[93]  Jillian H. Fecteau,et al.  Correlates of Capture of Attention and Inhibition of Return across Stages of Visual Processing , 2005, Journal of Cognitive Neuroscience.

[94]  Christos Dimitrakakis,et al.  TORCS, The Open Racing Car Simulator , 2005 .

[95]  Jillian H. Fecteau,et al.  Salience, relevance, and firing: a priority map for target selection , 2006, Trends in Cognitive Sciences.

[96]  Mohan M. Trivedi,et al.  Turn-Intent Analysis Using Body Pose for Intelligent Driver Assistance , 2006, IEEE Pervasive Computing.

[97]  Louis Tijerina,et al.  Driver Workload Metrics Task 2 Final Report , 2006 .

[98]  Mary Hayhoe,et al.  Control of attention and gaze in complex environments. , 2006, Journal of vision.

[99]  L Longtang Chen,et al.  Head movements evoked by electrical stimulation in the frontal eye field of the monkey: evidence for independent eye and head control. , 2006, Journal of neurophysiology.

[100]  Gin McCollum,et al.  Variables Contributing to the Coordination of Rapid Eye/Head Gaze Shifts , 2006, Biological Cybernetics.

[101]  Robert M. McPeek,et al.  Incomplete Suppression of Distractor-Related Activity in the Frontal Eye Field Results in Curved Saccades , 2006 .

[102]  M. Goldberg,et al.  Saccades, salience and attention: the role of the lateral intraparietal area in visual behavior. , 2006, Progress in brain research.

[103]  M. Hayhoe,et al.  Look-ahead fixations: anticipatory eye movements in natural tasks , 2007, Experimental Brain Research.

[104]  L. Itti,et al.  Visual causes versus correlates of attentional selection in dynamic scenes , 2006, Vision Research.

[105]  Etienne Olivier,et al.  Contribution of the Monkey Frontal Eye Field to Covert Visual Attention , 2006, The Journal of Neuroscience.

[106]  Mohan M. Trivedi,et al.  Driver Behavior and Situation Aware Brake Assistance for Intelligent Vehicles , 2007, Proceedings of the IEEE.

[107]  Neeraj J Gandhi,et al.  Role of the primate superior colliculus in the control of head movements. , 2007, Journal of neurophysiology.

[108]  Mohan M. Trivedi,et al.  Holistic Sensing and Active Displays for Intelligent Driver Support Systems , 2007, Computer.

[109]  Albert F Fuchs,et al.  Contribution of the frontal eye field to gaze shifts in the head-unrestrained monkey: effects of microstimulation. , 2007, Journal of neurophysiology.

[110]  Tarak Gandhi,et al.  Looking-In and Looking-Out of a Vehicle: Computer-Vision-Based Enhanced Vehicle Safety , 2007, IEEE Transactions on Intelligent Transportation Systems.

[111]  Laurent Itti,et al.  Beyond bottom-up: Incorporating task-dependent influences into a computational model of spatial attention , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[112]  Neeraj J Gandhi,et al.  Dissociation of eye and head components of gaze shifts by stimulation of the omnipause neuron region. , 2007, Journal of neurophysiology.

[113]  Mary M Hayhoe,et al.  Task and context determine where you look. , 2016, Journal of vision.

[114]  Jake K. Aggarwal,et al.  Hierarchical Recognition of Human Activities Interacting with Objects , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[115]  Mohan M. Trivedi,et al.  Head Pose Estimation for Driver Assistance Systems: A Robust Algorithm and Experimental Evaluation , 2007, 2007 IEEE Intelligent Transportation Systems Conference.

[116]  Brian D Corneil,et al.  Widespread presaccadic recruitment of neck muscles by stimulation of the primate frontal eye fields. , 2007, Journal of neurophysiology.

[117]  Michael L. Mack,et al.  VISUAL SALIENCY DOES NOT ACCOUNT FOR EYE MOVEMENTS DURING VISUAL SEARCH IN REAL-WORLD SCENES , 2007 .

[118]  A. Pentland Social Signal Processing [Exploratory DSP] , 2007, IEEE Signal Processing Magazine.

[119]  Linda Ng Boyle,et al.  Visual Attention in Driving: The Effects of Cognitive Load and Visual Disruption , 2007, Hum. Factors.

[120]  Mohan M. Trivedi,et al.  Lane Change Intent Analysis Using Robust Operators and Sparse Bayesian Learning , 2005, IEEE Transactions on Intelligent Transportation Systems.

[121]  Robert M McPeek,et al.  Reversal of a distractor effect on saccade target selection after superior colliculus inactivation. , 2008, Journal of neurophysiology.

[122]  Neeraj J Gandhi,et al.  Effect of reversible inactivation of superior colliculus on head movements. , 2008, Journal of neurophysiology.

[123]  Matteo Valsecchi,et al.  Head movements modulate visual responsiveness in the absence of gaze shifts , 2008, Neuroreport.

[124]  Brian D Corneil,et al.  Neuromuscular consequences of reflexive covert orienting , 2008, Nature Neuroscience.

[125]  Harold Pashler,et al.  Task prioritisation in multitasking during driving: opportunity to abort a concurrent task does not insulate braking responses from dual‐task slowing , 2008 .

[126]  E. Freedman Coordination of the eyes and head during visual orienting , 2008, Experimental Brain Research.

[127]  Mohan M. Trivedi,et al.  Simultaneous Eye Tracking and Blink Detection with Interactive Particle Filters , 2008, EURASIP J. Adv. Signal Process..

[128]  Tim K Marks,et al.  SUN: A Bayesian framework for saliency using natural statistics. , 2008, Journal of vision.

[129]  Kostas Karpouzis,et al.  Estimation of behavioral user state based on eye gaze and head pose—application in an e-learning environment , 2009, Multimedia Tools and Applications.

[130]  Mohan M. Trivedi,et al.  On the Roles of Eye Gaze and Head Dynamics in Predicting Driver's Intent to Change Lanes , 2009, IEEE Transactions on Intelligent Transportation Systems.

[131]  Mohan M. Trivedi,et al.  Head and gaze dynamics in visual attention and context learning , 2009, 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[132]  D. Ballard,et al.  Modelling the role of task in the control of gaze , 2009, Visual cognition.

[133]  Mohan M. Trivedi,et al.  Head Pose Estimation in Computer Vision: A Survey , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[134]  Gunnar Blohm,et al.  Differential influence of attention on gaze and head movements. , 2009, Journal of neurophysiology.

[135]  Mohan M. Trivedi,et al.  A Novel Active Heads-Up Display for Driver Assistance , 2009, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[136]  Nuno Vasconcelos,et al.  Spatiotemporal Saliency in Dynamic Scenes , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[137]  Look ahead fixations and visuo-motor planning , 2010 .

[138]  Thomas B. Moeslund,et al.  Long-Term Occupancy Analysis Using Graph-Based Optimisation in Thermal Imagery , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[139]  Joshua D. Cosman,et al.  The Control of Visual Attention , 2014 .