Probabilistic estimation of the driver's gaze from head orientation and position

Visual attention is an important factor when studying driver behavior. While the location of the pupil can provide accurate information about gaze, the vehicle environment poses challenges that prevent the use of off-the-shelf gaze detection algorithms in the car. Head pose can be used to approximate the driver's visual attention, providing a coarse estimate which can be good enough for many applications. However, the relation between head pose and gaze is not one-to-one, depending on the driver, cognitive load, and visual task. Instead of detecting a precise gaze direction, this paper proposes a novel approach which creates a probabilistic map describing visual attention. The approach relies on Gaussian process regression (GPR), which takes the position and orientation of the drivers' head to estimate the probability that the driver is looking at a given point. The approach creates confidence regions describing the most likely gaze directions. We evaluate the proposed approach with naturalistic recordings in real roads, where we estimate the position and orientation of the driver's head using a headband with fiducial markers. The experimental evaluation demonstrates that 89.4% of drivers' gaze are included in the 95% confidence region predicted by our model. The proposed system can provide valuable information for navigation, infotainment, safety and communication systems.

[1]  Raja Bala,et al.  Estimating Gaze Direction of Vehicle Drivers Using a Smartphone Camera , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[2]  Edwin Olson,et al.  AprilTag: A robust and flexible visual fiducial system , 2011, 2011 IEEE International Conference on Robotics and Automation.

[3]  Santokh Singh,et al.  Critical Reasons for Crashes Investigated in the National Motor Vehicle Crash Causation Survey , 2015 .

[4]  Qiang Ji,et al.  Real-Time Eye, Gaze, and Face Pose Tracking for Monitoring Driver Vigilance , 2002, Real Time Imaging.

[5]  W. Kabsch.  A discussion of the solution for the best rotation to relate two sets of vectors , 1978 .

[6]  Carlos Busso,et al.  Detecting Drivers' Mirror-Checking Actions and Its Application to Maneuver and Secondary Task Recognition , 2016, IEEE Transactions on Intelligent Transportation Systems.

[7]  Carlos Busso,et al.  Predicting Perceived Visual and Cognitive Distractions of Drivers With Multimodal Features , 2015, IEEE Transactions on Intelligent Transportation Systems.

[8]  Mohan M. Trivedi,et al.  On the Roles of Eye Gaze and Head Dynamics in Predicting Driver's Intent to Change Lanes , 2009, IEEE Transactions on Intelligent Transportation Systems.

[9]  Carlos Busso,et al.  Evaluating the robustness of an appearance-based gaze estimation method for multimodal interfaces , 2013, ICMI '13.

[10]  Kang Ryoung Park,et al.  Real-Time Gaze Estimator Based on Driver's Head Orientation for Forward Collision Warning System , 2011, IEEE Transactions on Intelligent Transportation Systems.

[11]  Reinhard Klette,et al.  Look at the Driver, Look at the Road: No Distraction! No Accident! , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Shumeet Baluja,et al.  Non-Intrusive Gaze Tracking Using Artificial Neural Networks , 1993, NIPS.

[13]  Jean Underwood,et al.  Visual attention while driving: sequences of eye fixations made by experienced and novice drivers , 2003, Ergonomics.

[14]  Carlos Busso,et al.  Analysis of facial features of drivers under cognitive and visual distractions , 2013, 2013 IEEE International Conference on Multimedia and Expo (ICME).

[15]  Jiann-Der Lee,et al.  A Fuzzy Rules-Based Driver Assistance System , 2015 .

[16]  Luke Fletcher,et al.  Driver Inattention Detection based on Eye Gaze—Road Event Correlation , 2009, Int. J. Robotics Res..

[17]  Mohan M. Trivedi,et al.  Robust and continuous estimation of driver gaze zone by dynamic analysis of multiple face videos , 2014, 2014 IEEE Intelligent Vehicles Symposium Proceedings.

[18]  Carlos Busso,et al.  Analyzing the relationship between head pose and gaze to model driver visual attention , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[19]  Peter Robinson,et al.  Constrained Local Neural Fields for Robust Facial Landmark Detection in the Wild , 2013, 2013 IEEE International Conference on Computer Vision Workshops.

[20]  A. Sathyanarayana,et al.  UTDrive: Driver Behavior and Speech Interactive Systems for In-Vehicle Environments , 2007, 2007 IEEE Intelligent Vehicles Symposium.

[21]  G H Robinson,et al.  Visual Search by Automobile Drivers , 1972, Human factors.

[22]  Carlos Busso,et al.  Modeling of Driver Behavior in Real World Scenarios Using Multiple Noninvasive Sensors , 2013, IEEE Transactions on Multimedia.

[23]  Carlos Busso,et al.  Challenges in head pose estimation of drivers in naturalistic recordings using existing tools , 2017, 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC).

[24]  M. Trivedi,et al.  Head and eye gaze dynamics during visual attention shifts in complex environments. , 2012, Journal of vision.

[25]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.