BubbleView

In this article, we present BubbleView, an alternative methodology for eye tracking using discrete mouse clicks to measure which information people consciously choose to examine. BubbleView is a mouse-contingent, moving-window interface in which participants are presented with a series of blurred images and click to reveal “bubbles” -- small, circular areas of the image at original resolution, similar to having a confined area of focus like the eye fovea. Across 10 experiments with 28 different parameter combinations, we evaluated BubbleView on a variety of image types: information visualizations, natural images, static webpages, and graphic designs, and compared the clicks to eye fixations collected with eye-trackers in controlled lab settings. We found that BubbleView clicks can both (i) successfully approximate eye fixations on different images, and (ii) be used to rank image and design elements by importance. BubbleView is designed to collect clicks on static images, and works best for defined tasks such as describing the content of an information visualization or measuring image importance. BubbleView data is cleaner and more consistent than related methodologies that use continuous mouse movements. Our analyses validate the use of mouse-contingent, moving-window methodologies as approximating eye fixations for different image and task types.

[1]  Frédo Durand,et al.  Learning Visual Importance for Graphic Designs and Data Visualizations , 2017, UIST.

[2]  Jorma Laaksonen,et al.  Saliency Revisited: Analysis of Mouse Movements Versus Fixations , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Frédo Durand,et al.  Where Should Saliency Models Look Next? , 2016, ECCV.

[4]  James Hays,et al.  WebGazer: Scalable Webcam Eye Tracking Using User Interactions , 2016, IJCAI.

[5]  Wojciech Matusik,et al.  Eye Tracking for Everyone , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  Dhruv Batra,et al.  Human Attention in Visual Question Answering: Do Humans and Deep Networks look at the same regions? , 2016, EMNLP.

[7]  Pingmei Xu,et al.  Spatio-Temporal Modeling and Prediction of Visual Attention in Graphical User Interfaces , 2016, CHI.

[8]  Frédo Durand,et al.  What Do Different Evaluation Metrics Tell Us About Saliency Models? , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  Noel E. O'Connor,et al.  Shallow and Deep Convolutional Networks for Saliency Prediction , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  Hanspeter Pfister,et al.  Beyond Memorability: Visualization Recognition and Recall , 2016, IEEE Transactions on Visualization and Computer Graphics.

[11]  Christof Lutteroth,et al.  Gaze vs. Mouse: A Fast and Accurate Gaze-Only Click Alternative , 2015, UIST.

[12]  E. M. DeGennaro,et al.  Towards the quantitative evaluation of visual attention models , 2015, Vision Research.

[13]  Michelle A. Borkin,et al.  Eye Fixation Metrics for Large Scale Evaluation and Comparison of Information Visualizations , 2015, ETVIS.

[14]  R. Venkatesh Babu,et al.  DeepFix: A Fully Convolutional Neural Network for Predicting Human Eye Fixations , 2015, IEEE Transactions on Image Processing.

[15]  Qi Zhao,et al.  Predicting Eye Fixations on Webpage With an Ensemble of Early Features and High-Level Representations from Deep Network , 2015, IEEE Transactions on Multimedia.

[16]  A. Veeraraghavan,et al.  TabletGaze: Unconstrained Appearance-based Gaze Estimation in Mobile Tablets , 2015, 1508.01244.

[17]  Qi Zhao,et al.  SALICON: Saliency in Context , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Ali Borji,et al.  CAT2000: A Large Scale Fixation Dataset for Boosting Saliency Research , 2015, ArXiv.

[19]  Pingmei Xu,et al.  TurkerGaze: Crowdsourcing Saliency with Webcam based Eye Tracking , 2015, ArXiv.

[20]  Krzysztof Z. Gajos,et al.  A Crowdsourced Alternative to Eye-tracking for Visualization Understanding , 2015, CHI Extended Abstracts.

[21]  Aaron Hertzmann,et al.  DesignScape: Design with Interactive Layout Suggestions , 2015, CHI.

[22]  Mario Fritz,et al.  Appearance-based gaze estimation in the wild , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Sören Preibusch,et al.  Privacy considerations for a pervasive eye tracking world , 2014, UbiComp Adjunct.

[24]  Aaron Hertzmann,et al.  Learning Layouts for Single-PageGraphic Designs , 2014, IEEE Transactions on Visualization and Computer Graphics.

[25]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[26]  Keith Rayner,et al.  The gaze-contingent moving window in reading: Development and review , 2014 .

[27]  Jean-Marc Odobez,et al.  EYEDIAP: a database for the development and evaluation of gaze estimation algorithms from RGB and RGB-D cameras , 2014, ETRA.

[28]  Jennifer C. Romano Bergstrom,et al.  Eye tracking in user experience design , 2014 .

[29]  Shenmin Zhang,et al.  What do saliency models predict? , 2014, Journal of vision.

[30]  Qi Zhao,et al.  Predicting human gaze beyond pixels. , 2014, Journal of vision.

[31]  Miad Faezipour,et al.  Eye Tracking and Head Movement Detection: A State-of-Art Survey , 2013, IEEE Journal of Translational Engineering in Health and Medicine.

[32]  Jonathan Krause,et al.  Fine-Grained Crowdsourcing for Fine-Grained Recognition , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Sung-Hee Kim,et al.  Does an Eye Tracker Tell the Truth about Visualizations?: Findings while Investigating Visualizations for Decision Making , 2012, IEEE Transactions on Visualization and Computer Graphics.

[34]  Thierry Baccino,et al.  Methods for comparing scanpaths and saliency maps: strengths and weaknesses , 2012, Behavior Research Methods.

[35]  Ryen W. White,et al.  User see, user point: gaze and cursor alignment in web search , 2012, CHI.

[36]  Frédo Durand,et al.  A Benchmark of Computational Models of Saliency to Predict Human Fixations , 2012 .

[37]  Kenneth Holmqvist,et al.  Eye tracking: a comprehensive guide to methods and measures , 2011 .

[38]  Peter König,et al.  Measures and Limits of Models of Fixation Selection , 2011, PloS one.

[39]  Ryan O. Murphy,et al.  Flashlight - Recording information acquisition online , 2011, Comput. Hum. Behav..

[40]  Eugene Agichtein,et al.  ViewSer: enabling large-scale remote user studies of web search examination and interaction , 2011, SIGIR.

[41]  Ryen W. White,et al.  No clicks, no problem: using cursor movements to understand and improve search , 2011, CHI.

[42]  D. Ballard,et al.  Eye guidance in natural vision: reinterpreting salience. , 2011, Journal of vision.

[43]  Eugene Agichtein,et al.  Towards predicting web searcher gaze position from mouse movements , 2010, CHI Extended Abstracts.

[44]  Jakob Nielsen,et al.  Eyetracking Web Usability , 2009 .

[45]  Frédo Durand,et al.  Learning to predict where humans look , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[46]  Stephan Diehl,et al.  Comparing the Readability of Graph Layouts using Eyetracking and Task-oriented Analysis , 2009, CAe.

[47]  Meredith Ringel Morris,et al.  What do you see when you're surfing?: using eye tracking to predict salient regions of web pages , 2009, CHI.

[48]  Aniket Kittur,et al.  Crowdsourcing user studies with Mechanical Turk , 2008, CHI.

[49]  Kerry Rodden,et al.  Eye-mouse coordination patterns on web search results pages , 2008, CHI Extended Abstracts.

[50]  Benjamin W Tatler,et al.  The central fixation bias in scene viewing: selecting an optimal viewing position independently of motor biases and image feature distributions. , 2007, Journal of vision.

[51]  Weidong Huang,et al.  Using eye tracking to investigate graph layout effects , 2007, 2007 6th International Asia-Pacific Symposium on Visualization.

[52]  Markku Tukiainen,et al.  Validating the Restricted Focus Viewer: A study using eye-movement tracking , 2007, Behavior research methods.

[53]  Edward Cutrell,et al.  What are you looking for?: an eye-tracking study of information usage in web search , 2007, CHI.

[54]  Bernhard Schölkopf,et al.  A Nonparametric Approach to Bottom-Up Visual Saliency , 2006, NIPS.

[55]  Peter Tarasewich,et al.  The Enhanced Restricted Focus Viewer , 2005, Int. J. Hum. Comput. Interact..

[56]  Markku Tukiainen,et al.  Effects of display blurring on the behavior of novices and experts during program debugging , 2005, CHI Extended Abstracts.

[57]  Iain D. Gilchrist,et al.  Visual correlates of fixation selection: effects of scale and time , 2005, Vision Research.

[58]  M. Hayhoe Advances in Relating Eye Movements and Cognition. , 2004, Infancy : the official journal of the International Society on Infant Studies.

[59]  Laura A. Dabbish,et al.  Labeling images with a computer game , 2004, AAAI Spring Symposium: Knowledge Collection from Volunteer Contributors.

[60]  Bing Pan,et al.  The determinants of web page viewing behavior: an eye-tracking study , 2004, ETRA.

[61]  Michael J. Spivey,et al.  Eye Movements and Problem Solving , 2003, Psychological science.

[62]  Lester C. Loschky,et al.  Gaze-Contingent Multiresolutional Displays: An Integrative Review , 2003, Hum. Factors.

[63]  Kim Marriott,et al.  A tool for tracking visual attention: The Restricted Focus Viewer , 2003, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[64]  Andrew T Duchowski,et al.  A breadth-first survey of eye-tracking applications , 2002, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[65]  Joseph H. Goldberg,et al.  Eye tracking in web search tasks: design implications , 2002, ETRA.

[66]  Michael E. Holmes,et al.  Visual attention to repeated internet images: testing the scanpath theory on the world wide web , 2002, ETRA.

[67]  Andrew J. Stewart,et al.  Integrating text and pictorial information: eye movements when looking at print advertisements. , 2001, Journal of experimental psychology. Applied.

[68]  Frédéric Gosselin,et al.  Bubbles: a technique to reveal the use of information in recognition tasks , 2001, Vision Research.

[69]  John R. Anderson,et al.  What can a mouse cursor tell us more?: correlation of eye/mouse movements on web browsing , 2001, CHI Extended Abstracts.

[70]  Alan F. Blackwell,et al.  Restricted Focus Viewer: A Tool for Tracking Visual Attention , 2000, Diagrams.

[71]  Joseph H. Goldberg,et al.  Computer interface evaluation using eye movements: methods and constructs , 1999 .

[72]  K. Rayner Eye movements in reading and information processing: 20 years of research. , 1998, Psychological bulletin.

[73]  D. Pomerleau,et al.  Non-Intrusive Gaze Tracking Using Artificial Neural Networks , 1993, NIPS.

[74]  W. Graf,et al.  Ergonomic evaluation of user-interfaces by means of eye-movement data , 1989 .

[75]  M. Just,et al.  Eye fixations and cognitive processes , 1976, Cognitive Psychology.

[76]  G. McConkie,et al.  The span of the effective stimulus during a fixation in reading , 1975 .

[77]  L. Stark,et al.  Scanpaths in saccadic eye movements while viewing and recognizing patterns. , 1971, Vision research.

[78]  Päivi Majaranta,et al.  Eye Tracking and Eye-Based Human–Computer Interaction , 2014 .

[79]  Qi Zhao,et al.  Webpage Saliency , 2014, ECCV.

[80]  Ali Borji,et al.  Quantitative Analysis of Human-Model Agreement in Visual Saliency Modeling: A Comparative Study , 2013, IEEE Transactions on Image Processing.

[81]  Ali Borji,et al.  State-of-the-Art in Visual Attention Modeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[82]  Ruth Rosenholtz,et al.  Do predictions of visual perception aid design? , 2011, TAP.

[83]  Ronald A. Rensink Human Attention in Digital Environments: The management of visual attention in graphic displays , 2011 .

[84]  White Tobii Eye Tracking An introduction to eye tracking and Tobii Eye Trackers , 2010 .

[85]  Henrik I. Christensen,et al.  Computational visual attention systems and their cognitive foundations: A survey , 2010, TAP.

[86]  James T. Miller,et al.  How Users View Web Pages: An Exploration of Cognitive and Perceptual Mechanisms , 2007 .

[87]  Linden J. Ball,et al.  Eye tracking in HCI and usability research. , 2006 .

[88]  Ma Sasse,et al.  The Eyes Never Lie: The Use of Eyetracking Data in HCI Research , 2002, CHI 2002.

[89]  Robert J. K. Jacob,et al.  Eye tracking in human-computer interaction and usability research : Ready to deliver the promises , 2002 .

[90]  Linden J. Ball,et al.  An Eye Movement Analysis of Web Page Usability , 2002 .

[91]  Derrick J. Parkhurst,et al.  Modeling the role of salience in the allocation of overt visual attention , 2002, Vision Research.

[92]  Eileen Kowler The role of visual and cognitive processes in the control of eye movement. , 1990, Reviews of oculomotor research.