Capturing real-world gaze behaviour: live and unplugged

Understanding human gaze behaviour has benefits from scientific understanding to many application domains. Current practices constrain possible use cases, requiring experimentation restricted to a lab setting or controlled environment. In this paper, we demonstrate a flexible unconstrained end-to-end solution that allows for collection and analysis of gaze data in real-world settings. To achieve these objectives, rich 3D models of the real world are derived along with strategies for associating experimental eye-tracking data with these models. In particular, we demonstrate the strength of photogrammetry in allowing these capabilities to be realized, and demonstrate the first complete solution for 3D gaze analysis in large-scale outdoor environments using standard camera technology without fiducial markers. The paper also presents techniques for quantitative analysis and visualization of 3D gaze data. As a whole, the body of techniques presented provides a foundation for future research, with new opportunities for experimental studies and computational modeling efforts.

[1]  Daniel Cremers,et al.  LSD-SLAM: Large-Scale Direct Monocular SLAM , 2014, ECCV.

[2]  Pierre Moulon,et al.  Python Photogrammetry Toolbox: A free solution for Three-Dimensional Documentation , 2011 .

[3]  Cyrill Stachniss,et al.  Simultaneous Localization and Mapping , 2016, Springer Handbook of Robotics, 2nd Ed..

[4]  Juan D. Tardós,et al.  ORB-SLAM2: An Open-Source SLAM System for Monocular, Stereo, and RGB-D Cameras , 2016, IEEE Transactions on Robotics.

[5]  Johann Schrammel,et al.  3D attention: measurement of visual saliency using eye tracking glasses , 2013, CHI Extended Abstracts.

[6]  Fiora Pirri,et al.  A3D: A Device for Studying Gaze in 3D , 2016, ECCV Workshops.

[7]  Yifan Peng,et al.  Studying Relationships between Human Gaze, Description, and Computer Vision , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[8]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Thies Pfeiffer,et al.  Model-based real-time visualization of realistic three-dimensional heat maps for mobile eye tracking and eye tracking in virtual reality , 2016, ETRA.

[10]  Jacob L. Orquin,et al.  Attention and choice: a review on eye movements in decision making. , 2013, Acta psychologica.

[11]  Kristina Höök,et al.  CHI '12 Extended Abstracts on Human Factors in Computing Systems , 2012, CHI 2012.

[12]  S. Tipper,et al.  Spread of inhibition across an object's surface , 1999 .

[13]  Thomas Driemeyer.  Rendering with mental ray® (mental ray® Handbooks) , 2005 .

[14]  D. Hubel,et al.  The role of fixational eye movements in visual perception , 2004, Nature Reviews Neuroscience.

[15]  Florian Heimerl,et al.  ISeeCube: visual analysis of gaze data for video , 2014, ETRA.

[16]  G. Crooks.  On Measures of Entropy and Information , 2015 .

[17]  J. Liu.  Simple technique for measurements of pulsed Gaussian-beam spot sizes. , 1982, Optics letters.

[18]  Raimund Dachselt,et al.  3D attentional maps: aggregated gaze visualizations in three-dimensional virtual environments , 2010, AVI.

[19]  Huaiyu Zhu.  On Information and Sufficiency , 1997 .

[20]  Thies Pfeiffer,et al.  GPU-accelerated attention map generation for dynamic 3D scenes , 2015, 2015 IEEE Virtual Reality (VR).

[21]  Thies Pfeiffer,et al.  EyeSee3D: a low-cost approach for analyzing mobile 3D eye tracking data using computer vision and augmented reality technology , 2014, ETRA.

[22]  Changchang Wu,et al.  SiftGPU : A GPU Implementation of Scale Invariant Feature Transform (SIFT) , 2007 .

[23]  Mehrdad Jazayeri,et al.  The spread of attention across features of a surface. , 2013, Journal of neurophysiology.

[24]  Javier Civera,et al.  Inverse Depth Parametrization for Monocular SLAM , 2008, IEEE Transactions on Robotics.

[25]  Peter Kovesi,et al.  Good Colour Maps: How to Design Them , 2015, ArXiv.

[26]  Richard Szeliski,et al.  Towards Internet-scale multi-view stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[27]  Anders Gustafsson,et al.  Consumer perception at the Point-of-Purchase : Evaluating proposed package designs in the eye-tracking lab , 2008 .

[28]  Roland Hess,et al.  The Essential Blender: Guide to 3D Creation with the Open Source Suite Blender , 2007 .

[29]  J. Beskow.  Eye Gaze Analysis in Human-Human Interactions , 2007 .

[30]  Thies Pfeiffer,et al.  EyeSee3D 2.0: model-based real-time analysis of mobile eye-tracking in static and dynamic three-dimensional scenes , 2016, ETRA.

[31]  Markus H. Gross,et al.  Algebraic point set surfaces , 2007, ACM Trans. Graph..

[32]  Cathleen M Moore,et al.  The spread of attention to hidden portions of occluded surfaces , 2005, Psychonomic bulletin & review.

[33]  Renata Bagdžiūnaitė,et al.  Close-range photogrammetry enables documentation of environment-induced deformation of architectural heritage , 2015 .

[34]  Marco Dubbini,et al.  Using Unmanned Aerial Vehicles (UAV) for High-Resolution Reconstruction of Topography: The Structure from Motion Approach on Coastal Environments , 2013, Remote. Sens..

[35]  Dariush Derakhshani.  Introducing Autodesk Maya 2012 , 2010 .