Should I Interrupt or Not?: Understanding Interruptions in Head-Mounted Display Settings

Head-mounted displays (HMDs) are being used for VR and AR applications and increasingly permeate our everyday life. At the same time, a detailed understanding of interruptions in settings where people wearing an HMD (HMD user) and people not wearing an HMD (bystander) is missing. We investigate (a) whether bystanders are capable of identifying when HMD users switch tasks by observing their gestures, and hence exploit opportune moments for interruptions, and (b) which strategies bystanders employ. In a lab study (N=64) we found that bystanders are able to successfully identify both task switches (83%) and tasks (77%) within only a few seconds of the task switch. Furthermore, we identified interruption strategies of bystanders. From our results we derive implications meant to support designers and practitioners in building HMD applications that are used in a co-located collaborative setting.

[1]  Abraham Bernstein,et al.  A scenario-based approach for direct interruptability prediction on wearable devices , 2007, Int. J. Pervasive Comput. Commun..

[2]  Brian P. Bailey,et al.  Leveraging characteristics of task structure to predict the cost of interruption , 2006, CHI.

[3]  Hai-Ning Liang,et al.  An exploration of usable authentication mechanisms for virtual reality systems , 2016, 2016 IEEE Asia Pacific Conference on Circuits and Systems (APCCAS).

[4]  Robert E. Kraut,et al.  Controlling interruptions: awareness displays and social motivation for coordination , 2004, CSCW.

[5]  Julian Frommel,et al.  Evaluating VR Driving Simulation from a Player Experience Perspective , 2017, CHI Extended Abstracts.

[6]  Wendy A. Kellogg,et al.  "I'd be overwhelmed, but it's just one more thing to do": availability and interruption in research management , 2002, CHI.

[7]  Harpreet Sareen,et al.  FaceDisplay: Towards Asymmetric Multi-User Interaction for Nomadic Virtual Reality , 2018, CHI.

[8]  Julian Frommel,et al.  ShareVR: Enabling Co-Located Experiences for Virtual Reality between HMD and Non-HMD Users , 2017, CHI.

[9]  Yichao Huang,et al.  A Pointing Gesture Based Egocentric Interaction System: Dataset, Approach and Application , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[10]  Franco Tecchia,et al.  Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks , 2015, Interact. Learn. Environ..

[11]  Doug A. Bowman,et al.  Testbed evaluation of navigation and text display techniques in an information-rich virtual environment , 2004, IEEE Virtual Reality 2004.

[12]  Thierry Duval,et al.  Vishnu: virtual immersive support for HelpiNg users an interaction paradigm for collaborative remote guiding in mixed reality , 2016, 2016 IEEE Third VR International Workshop on Collaborative Virtual Environments (3DCVE).

[13]  H. Hussmann,et al.  Seamless and Secure VR: Adapting and Evaluating Established Authentication Systems for Virtual Reality , 2017 .

[14]  Brian P. Bailey,et al.  Investigating the effectiveness of mental workload as a predictor of opportune moments for interruption , 2005, CHI Extended Abstracts.

[15]  Tomasz Mazuryk,et al.  Virtual Reality History , Applications , Technology and Future , 1999 .

[16]  Maria V. Sanchez-Vives,et al.  From presence to consciousness through virtual reality , 2005, Nature Reviews Neuroscience.

[17]  Boyang Li,et al.  Reducing Interruptions at Work: A Large-Scale Field Study of FlowLight , 2017, CHI.

[18]  Shaoting Zhang,et al.  Towards mobile immersive analysis: A study of applications , 2016, 2016 Workshop on Immersive Analytics (IA).

[19]  Jörg Müller,et al.  Cuenesics: using mid-air gestures to select items on interactive public displays , 2014, MobileHCI '14.

[20]  Mel Slater,et al.  Immersion, presence and performance in virtual environments: an experiment with tri-dimensional chess , 1996, VRST.

[21]  Brian P. Bailey,et al.  If not now, when?: the effects of interruption at different moments within task execution , 2004, CHI.

[22]  Mary Jo Hatch,et al.  Physical Barriers, Task Characteristics, and Interaction Activity in Research and Development Firms , 1987 .

[23]  James Fogarty,et al.  Biases in human estimation of interruptibility: effects and implications for practice , 2007, CHI.

[24]  M. McDaniel,et al.  (www.interscience.wiley.com) DOI: 10.1002/acp.1002 Delaying Execution of Intentions: Overcoming the Costs of Interruptions , 2022 .

[25]  Silke Geisen.  Augmented Reality in surgery , 2005 .

[26]  Graham Button,et al.  Published in: Technology in Working Order: Studies of Work, Interaction, and Technology, , 2022 .

[27]  Christian Heath,et al.  Disembodied Conduct: Interactional Asymmetries in Video-Mediated Communication , 1993 .

[28]  Andrew L. Kun,et al.  Calling while driving: an initial experiment with hololens , 2017 .

[29]  Susan Goldin-Meadow,et al.  Turkish- and English-speaking children display sensitivity to perceptual context in the referring expressions they produce in speech and gesture , 2012, Language and cognitive processes.

[30]  Karsten Rexroth,et al.  Augmented Reality in der Stadt: Neue Potenziale durch die Entwicklung einer Lichtmarker-Technologie , 2017 .

[31]  James Fogarty,et al.  Toolkit support for developing and deploying sensor-based statistical models of human situations , 2007, CHI.

[32]  B. Karsh,et al.  Interruptions and Distractions in Healthcare: Review and Reappraisal Method Inclusion and Exclusion Criteria Nih Public Access , 2022 .

[33]  Nadine B. Sarter,et al.  Not Now! Supporting Interruption Management by Indicating the Modality and Urgency of Pending Tasks , 2004, Hum. Factors.

[34]  Radu-Daniel Vatavu,et al.  User-defined gestures for free-hand TV control , 2012, EuroITV.

[35]  Tobias Höllerer,et al.  Gesture-based augmented reality annotation , 2017, 2017 IEEE Virtual Reality (VR).

[36]  Eric Horvitz,et al.  Learning and reasoning about interruption , 2003, ICMI '03.

[37]  Adam Kendon.  Introduction to the Special issue: Gesture and Understanding in Interaction , 1994 .

[38]  Mohammad Shahidul Hasan,et al.  Innovative developments in HCI and future trends , 2015, 2015 21st International Conference on Automation and Computing (ICAC).

[39]  Søren S. Sørensen.  The Development of Augmented Reality as a Tool in Architectural and Urban Design , 2013 .

[40]  Niels Henze,et al.  Reading on Smart Glasses: The Effect of Text Position, Presentation Type and Walking , 2018, CHI.

[41]  Darren Newtson.  Attribution and the unit of perception of ongoing behavior. , 1973 .

[42]  Paul Hockett,et al.  Augmented Reality with Hololens: Experiential Architectures Embedded in the Real World , 2016, ArXiv.

[43]  Christopher G. Atkeson,et al.  Predicting human interruptibility with sensors: a Wizard of Oz feasibility study , 2003, CHI '03.

[44]  Mohamed Khamis,et al.  TransparentHMD: revealing the HMD user's face to bystanders , 2017, MUM.

[45]  Daniel A. Guttentag.  Virtual reality: Applications and implications for tourism , 2010 .

[46]  Glyn Lawson,et al.  Future directions for the development of virtual reality within an automotive manufacturer. , 2016, Applied ergonomics.

[47]  Stuart Reeves,et al.  Ways of Spectating: Unravelling Spectator Participation in Kinect Play , 2017, CHI.

[48]  Jörg Müller,et al.  MyPosition: sparking civic discourse by a public interactive poll visualization , 2014, CSCW.

[49]  James Fogarty,et al.  Examining the robustness of sensor-based statistical models of human interruptibility , 2004, CHI.

[50]  Shinsaku Kiyomoto,et al.  PrivacyGuide: Towards an Implementation of the EU GDPR on Internet Privacy Policy Evaluation , 2018, IWSPA@CODASPY.

[51]  Joyce Ho,et al.  Using context-aware computing to reduce the perceived burden of interruptions from mobile devices , 2005, CHI.

[52]  Oliver Brdiczka,et al.  Learning to detect user activity and availability from a variety of sensor data , 2004, Second IEEE Annual Conference on Pervasive Computing and Communications, 2004. Proceedings of the.

[53]  D. Norman,et al.  Psychological Issues in Support of Multiple Activities , 1986 .

[54]  Nassir Navab,et al.  mirracle: An augmented reality magic mirror system for anatomy education , 2012, 2012 IEEE Virtual Reality Workshops (VRW).

[55]  Takahiro Tanaka,et al.  Study of user interruptibility estimation based on focused application switching , 2011, CSCW '11.

[56]  Michael J. Singer,et al.  Measuring Presence in Virtual Environments: A Presence Questionnaire , 1998, Presence.

[57]  J. Edward Swan,et al.  Peripheral visual information and its effect on distance judgments in virtual and augmented environments , 2011, APGV '11.

[58]  Manfred Tscheligi,et al.  Mid-air Authentication Gestures: An Exploration of Authentication Based on Palm and Finger Motions , 2014, ICMI.

[59]  Tomasz Mazuryk,et al.  Zoom Rendering: Improving 3-D Rendering Performance with 2-D Operations , 1996 .

[60]  M. Argyle,et al.  A Cross-Cultural Study of the Communication of Extra-Verbal Meaning by Gesture , 1975 .

[61]  Tadayoshi Kohno,et al.  In situ with bystanders of augmented reality glasses: perspectives on recording and privacy-mediating technologies , 2014, CHI.

[62]  Alan Wexelblat.  Research Challenges in Gesture: Open Issues and Unsolved Problems , 1997, Gesture Workshop.

[63]  Florian Alt,et al.  Seamless and Secure VR: Adapting and Evaluating Established Authentication Systems for Virtual Reality , 2017 .

[64]  Radu-Daniel Vatavu,et al.  Audience Silhouettes: Peripheral Awareness of Synchronous Audience Kinesics for Social Television , 2015, TVX.

[65]  Roderick Murray-Smith,et al.  A Dose of Reality: Overcoming Usability Challenges in VR Head-Mounted Displays , 2015, CHI Extended Abstracts.

[66]  Kaisa Väänänen,et al.  Gesture Driven Interaction as a Human Factor in Virtual Environments - An Approach with Neural Networks , 1993, Virtual Reality Systems.

[67]  A. Lecours,et al.  The Biological foundations of gestures : motor and semiotic aspects , 1986 .

[68]  Sriram Subramanian,et al.  Would you do that?: understanding social acceptance of gestural interfaces , 2010, Mobile HCI.

[69]  Amit Garg,et al.  Designing intentional impossible spaces in virtual reality narratives: A case study , 2017, 2017 IEEE Virtual Reality (VR).

[70]  Heinrich Hußmann,et al.  Intelligent Interruptions for IVR: Investigating the Interplay between Presence, Workload and Attention , 2018, CHI Extended Abstracts.

[71]  Ivan Poupyrev,et al.  The MagicBook: a transitional AR interface , 2001, Comput. Graph..

[72]  Mathias Beike,et al.  The Art Of Human Computer Interface Design , 2016 .

[73]  Jakob Nielsen,et al.  Heuristic evaluation of user interfaces , 1990, CHI '90.

[74]  A. J. Rivera,et al.  A socio-technical systems approach to studying interruptions: understanding the interrupter's perspective. , 2014, Applied ergonomics.

[75]  F Zorriassatine,et al.  A survey of virtual prototyping techniques for mechanical product development , 2003 .

[76]  Stuart M. Allen,et al.  Interruptibility prediction for ubiquitous systems: conventions and new directions from a growing field , 2015, UbiComp.

[77]  Miss Bhavana Borkar,et al.  4D Password Mechanism , 2016 .

[78]  Markus Funk,et al.  HoloCollab: a shared virtual platform for physical assembly training using spatially-aware head-mounted displays , 2017, IOT.