Gaze-Guided Narratives: Adapting Audio Guide Content to Gaze in Virtual and Real Environments

Exploring a city panorama from a vantage point is a popular tourist activity. Typical audio guides that support this activity are limited by their lack of responsiveness to user behavior and by the difficulty of matching audio descriptions to the panorama. These limitations can inhibit the acquisition of information and negatively affect user experience. This paper proposes Gaze-Guided Narratives as a novel interaction concept that helps tourists find specific features in the panorama (gaze guidance) while adapting the audio content to what has been previously looked at (content adaptation). Results from a controlled study in a virtual environment (n=60) revealed that a system featuring both gaze guidance and content adaptation obtained better user experience, lower cognitive load, and led to better performance in a mapping task compared to a classic audio guide. A second study with tourists situated at a vantage point (n=16) further demonstrated the feasibility of this approach in the real world.

[1]  Andreas Bulling,et al.  Pupil: an open source platform for pervasive eye tracking and mobile gaze-based interaction , 2014, UbiComp Adjunct.

[2]  Otmar Hilliges,et al.  Learning to find eye region landmarks for remote gaze estimation in unconstrained settings , 2018, ETRA.

[3]  Dan Witzner Hansen,et al.  Eye tracking in the wild , 2005, Comput. Vis. Image Underst..

[4]  Wolfgang Rosenstiel,et al.  The art of pervasive eye tracking: unconstrained eye tracking in the Austrian Gallery Belvedere , 2018, PETMEI@ETRA.

[5]  Ann McNamara,et al.  Directing gaze in narrative art , 2012, SAP.

[6]  Lucas Paletta,et al.  Mobile Augmented Reality for Tourists - MARFT , 2012 .

[7]  Thomas Kieninger,et al.  Museum Guide 2.0 – An Eye-Tracking based Personal Assistant for Museums and Exhibits , 2011 .

[8]  Andrew T. Duchowski,et al.  PETMEI 2011: the 1st international workshop on pervasive eye tracking and mobile eye-based interaction , 2011, UbiComp '11.

[9]  Andrew T. Duchowski,et al.  Audio description as an aural guide of children's visual attention: evidence from an eye-tracking study , 2012, ETRA.

[10]  Bill Albert,et al.  Chapter 6 – Self-Reported Metrics , 2013 .

[11]  Martin Schrepp,et al.  Construction and Evaluation of a User Experience Questionnaire , 2008, USAB.

[12]  Stephen A. Brewster,et al.  Multimodal 'eyes-free' interaction techniques for wearable devices , 2003, CHI '03.

[13]  Florian Alt,et al.  EyeScout: Active Eye Tracking for Position and Movement Independent Gaze Interaction with Large Public Displays , 2017, UIST.

[14]  Roope Raisamo,et al.  Gaze gestures and haptic feedback in mobile devices , 2014, CHI.

[15]  Jan O. Borchers,et al.  Simplifying orientation measurement for mobile audio augmented reality applications , 2014, CHI.

[16]  Gerhard Tröster,et al.  Eye Movement Analysis for Activity Recognition Using Electrooculography , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Kenny R. Coventry,et al.  Spatial Language and Dialogue , 2009, Explorations in language and space.

[18]  Keith Cheverst,et al.  Experiences of developing and deploying a context-aware tourist guide: the GUIDE project , 2000, MobiCom '00.

[19]  Konrad Schindler,et al.  Gaze-Informed location-based services , 2017, Int. J. Geogr. Inf. Sci..

[20]  Johannes Schöning,et al.  NatureCHI: Unobtrusive User Experiences with Technology in Nature , 2016, CHI Extended Abstracts.

[21]  Joseph H. Goldberg,et al.  Identifying fixations and saccades in eye-tracking protocols , 2000, ETRA.

[22]  Thies Pfeiffer,et al.  Guiding visual search tasks using gaze-contingent auditory feedback , 2014, UbiComp Adjunct.

[23]  Rafael Ballagas,et al.  Gaming Tourism: Lessons from Evaluating REXplorer, a Pervasive Game for Tourists , 2009, Pervasive.

[24]  Maha Aly,et al.  Empathy Up , 2017, CHI Extended Abstracts.

[25]  Thies Pfeiffer,et al.  Attention guiding techniques using peripheral vision and eye tracking for feedback in augmented-reality-based assistance systems , 2017, 2017 IEEE Symposium on 3D User Interfaces (3DUI).

[26]  Blair MacIntyre,et al.  Exploring spatial narratives and mixed reality experiences in Oakland Cemetery , 2005, ACE '05.

[27]  Christoph Schlieder,et al.  Starting to get bored: an outdoor eye tracking study of tourists exploring a city panorama , 2014, ETRA.

[28]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[29]  Johannes Schöning,et al.  Evaluating automatically generated location-based stories for tourists , 2008, CHI Extended Abstracts.

[30]  Luciano Gamberini,et al.  A two-step, user-centered approach to personalized tourist recommendations , 2017, CHItaly.

[31]  Brian F. Goldiez,et al.  Effects of Augmented Reality Display Settings on Human Wayfinding Performance , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[32]  Päivi Majaranta,et al.  Eye Tracking and Eye-Based Human–Computer Interaction , 2014 .

[33]  Alan Kennedy,et al.  Book Review: Eye Tracking: A Comprehensive Guide to Methods and Measures , 2016, Quarterly journal of experimental psychology.

[34]  Gerhard Rigoll,et al.  GazeEverywhere: Enabling Gaze-only User Interaction on an Unmodified Desktop PC in Everyday Scenarios , 2017, CHI.

[35]  Gregory D. Abowd,et al.  Cyberguide: A mobile context‐aware tour guide , 1997, Wirel. Networks.

[36]  Daniela Petrelli,et al.  Voices from the War: Design as a Means of Understanding the Experience of Visiting Heritage , 2016, CHI.

[37]  J. B. Brooke,et al.  SUS: A 'Quick and Dirty' Usability Scale , 1996 .

[38]  Keith Cheverst,et al.  Encouraging Visitor Engagement and Reflection with the Landscape of the English Lake District:Exploring the potential of Locative Media , 2016 .

[39]  Martin Raubal,et al.  An Integrated Learning Management System for Location-Based Mobile Learning. , 2015 .

[40]  Wenping Wang,et al.  Close to the Action: Eye-Tracking Evaluation of Speaker-Following Subtitles , 2017, CHI.

[41]  Shumin Zhai,et al.  RealTourist - A Study of Augmenting Human-Human and Human-Computer Dialogue with Eye-Gaze Overlay , 2005, INTERACT.

[42]  Albrecht Schmidt,et al.  Implicit human computer interaction through context , 2000, Personal Technologies.

[43]  Marc Cavazza,et al.  Narratology for Interactive Storytelling: A Critical Introduction , 2006, TIDSE.

[44]  S. Hart,et al.  Development of NASA-TLX (Task Load Index): Results of Empirical and Theoretical Research , 1988 .

[45]  Alex Fridman,et al.  What Can Be Predicted from Six Seconds of Driver Glances? , 2016, CHI.

[46]  Martin Pielot,et al.  PocketNavigator: studying tactile navigation systems in-situ , 2012, CHI.

[47]  Anthony E. Richardson,et al.  Development of a self-report measure of environmental spatial ability. , 2002 .

[48]  Hong Va Leong,et al.  ScreenGlint: Practical, In-situ Gaze Estimation on Smartphones , 2017, CHI.

[49]  Ann McNamara,et al.  Subtle gaze direction , 2009, TOGS.

[50]  Dai-In Han,et al.  Dublin AR: Implementing Augmented Reality in Tourism , 2014, ENTER.

[51]  Andrew T. Duchowski,et al.  Discerning Ambient/Focal Attention with Coefficient K , 2016, ACM Trans. Appl. Percept..

[52]  W. Mackaness,et al.  Lecture Notes in Geoinformation and Cartography , 2006 .

[53]  Kenneth E. Foote,et al.  Historical space as narrative medium: on the configuration of spatial narratives of time at historical sites , 2008 .