ModularHMD: A Reconfigurable Mobile Head-Mounted Display Enabling Ad-hoc Peripheral Interactions with the Real World

We propose ModularHMD, a new mobile head-mounted display concept, which adopts a modular mechanism and allows a user to perform ad-hoc peripheral interaction with real-world devices or people during VR experiences. ModularHMD is comprised of a central HMD and three removable module devices installed in the periphery of the HMD cowl. Each module has four main states: occluding, extended VR view, video see-through (VST), and removed/reused. Among different combinations of module states, a user can quickly setup the necessary HMD forms, functions, and real-world visions for ad-hoc peripheral interactions without removing the headset. For instance, an HMD user can see her surroundings by switching a module into the VST mode. She can also physically remove a module to obtain direct peripheral visions of the real world. The removed module can be reused as an instant interaction device (e.g., touch keyboards) for subsequent peripheral interactions. Users can end the peripheral interaction and revert to a full VR experience by re-mounting the module. We design ModularHMD’s configuration and peripheral interactions with real-world objects and people. We also implement a proof-of-concept prototype of ModularHMD to validate its interactions capabilities through a user study. Results show that ModularHMD is an effective solution that enables both immersive VR and ad-hoc peripheral interactions.

[1]  Robert Xiao,et al.  Augmenting the Field-of-View of Head-Mounted Displays with Sparse Peripheral Displays , 2016, CHI.

[2]  Anderson Maciel,et al.  Characterizing Asymmetric Collaborative Interactions in Virtual and Augmented Realities , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[3]  Jörg Müller,et al.  ReconViguRation: Reconfiguring Physical Keyboards in Virtual Reality , 2019, IEEE Transactions on Visualization and Computer Graphics.

[4]  David A. Forsyth,et al.  Where's My Drink? Enabling Peripheral Real World Interactions While Using HMDs , 2015, ArXiv.

[5]  E. Hall,et al.  The Hidden Dimension , 1970 .

[6]  Roshan Lalintha Peiris,et al.  FacePush: Introducing Normal Force on Face with Head-Mounted Displays , 2018, UIST.

[7]  Mark Mine,et al.  Making VR work: building a real-world immersive modeling application in the virtual world , 2014, SUI.

[8]  Liwei Chan,et al.  FaceWidgets: Exploring Tangible Interaction on Face with Head-Mounted Displays , 2019, UIST.

[9]  Per Ola Kristensson,et al.  Effects of Hand Representations for Typing in Virtual Reality , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[10]  Yang-Sheng Chen,et al.  Visualizing the keyboard in virtual reality for enhancing immersive experience , 2017, SIGGRAPH Posters.

[11]  Kouta Minamizawa,et al.  TeleSight: enabling asymmetric collaboration in VR between HMD user and Non-HMD users , 2019, SIGGRAPH Emerging Technologies.

[12]  Julie R. Williamson,et al.  PlaneVR: Social Acceptability of Virtual Reality for Aeroplane Passengers , 2019, CHI.

[13]  Yuta Sugiura,et al.  AffectiveHMD: facial expression recognition in head mounted display using embedded photo reflective sensors , 2019, SIGGRAPH Emerging Technologies.

[14]  Ronald Azuma,et al.  ThinVR: Heterogeneous microlens arrays for compact, 180 degree FOV VR near-eye displays , 2020, IEEE Transactions on Visualization and Computer Graphics.

[15]  Niels Henze,et al.  Physical Keyboards in Virtual Reality: Analysis of Typing Performance and Effects of Avatar Hands , 2018, CHI.

[16]  R. W. Root,et al.  Informal Communication in Organizations: Form, Function, and Technology , 1990 .

[17]  Yoshifumi Kitamura,et al.  A Reconfigurable Mobile Head-Mounted Display Supporting Real World Interactions , 2021, CHI Extended Abstracts.

[18]  Per Ola Kristensson,et al.  Towards a Practical Virtual Office for Mobile Knowledge Workers , 2020, ArXiv.

[19]  Kasper Hornbæk,et al.  The Dream is Collapsing: The Experience of Exiting VR , 2018, CHI.

[20]  Jun Rekimoto,et al.  Deep dive: deep-neural-network-based video extension for immersive head-mounted display experiences , 2019, PerDis.

[21]  Ata Tara,et al.  Measuring magnitude of change by high-rise buildings in visual amenity conflicts in Brisbane , 2021 .

[22]  Julian Frommel,et al.  ShareVR: Enabling Co-Located Experiences for Virtual Reality between HMD and Non-HMD Users , 2017, CHI.

[23]  Eyal Ofek,et al.  RealityCheck: Blending Virtual Environments with Situated Physical Reality , 2019, CHI.

[24]  HMD Light: Sharing In-VR Experience via Head-Mounted Projector for Asymmetric Interaction , 2020, UIST.

[25]  Mitsunori Tada,et al.  Dollhouse VR: a multi-view, multi-user collaborative design workspace with VR technology , 2015, SIGGRAPH Asia Posters.

[26]  Per Ola Kristensson,et al.  Breaking the Screen: Interaction Across Touchscreen Boundaries in Virtual Reality for Mobile Knowledge Workers , 2020, IEEE Transactions on Visualization and Computer Graphics.

[27]  Kensuke Harada,et al.  Obstacle Avoidance Method in Real Space for Virtual Reality Immersion , 2018, 2018 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[28]  Evan Suma Rosenberg,et al.  Comparison of Audio and Visual Cues to Support Remote Guidance in Immersive Environments , 2020, ICAT-EGVE.

[29]  Christian Holz,et al.  SurfaceConstellations: A Modular Hardware Platform for Ad-Hoc Reconfigurable Cross-Device Workspaces , 2018, CHI.

[30]  I. Scott MacKenzie,et al.  Phrase sets for evaluating text entry techniques , 2003, CHI Extended Abstracts.

[31]  Ian P. Howard,et al.  Binocular Vision and Stereopsis , 1996 .

[32]  Jens Grubert,et al.  HeadPhones: Ad Hoc Mobile Multi-Display Environments through Head Tracking , 2017, CHI.

[33]  Jun Rekimoto,et al.  Behind-the-mask: a face-through head-mounted display , 2018, AVI.

[34]  Michael Lankes,et al.  The Virtual House of Medusa: Guiding Museum Visitors Through a Co-located Mixed Reality Installation , 2018, JCSG.

[35]  Christian Früh,et al.  Headset removal for virtual and mixed reality , 2017, SIGGRAPH Talks.

[36]  Mark S. Hancock,et al.  "Transport Me Away": Fostering Flow in Open Offices through Virtual Reality , 2020, CHI.

[37]  Akira Ishii,et al.  Let Your World Open: CAVE-based Visualization Methods of Public Virtual Reality towards a Shareable VR Experience , 2019, AH.

[38]  Mohamed Khamis,et al.  TransparentHMD: revealing the HMD user's face to bystanders , 2017, MUM.

[39]  Robert W. Lindeman,et al.  MDI: A Multi-channel Dynamic Immersion Headset for Seamless Switching between Virtual and Real World Activities , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[40]  Daniel Vogel,et al.  A Modular Smartphone for Lending , 2017, UIST.

[41]  Hans-Werner Gellersen,et al.  Substitutional Reality: Using the Physical Environment to Design Virtual Reality Experiences , 2015, CHI.

[42]  Daniel Sonntag,et al.  ModulAR: Eye-Controlled Vision Augmentations for Head Mounted Displays , 2015, IEEE Transactions on Visualization and Computer Graphics.

[43]  Stephen DiVerdi,et al.  TransceiVR: Bridging Asymmetrical Communication Between VR Users and External Collaborators , 2020, UIST.

[44]  Per Ola Kristensson,et al.  Text Entry in Immersive Head-Mounted Display-Based Virtual Reality Using Standard Keyboards , 2018, 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[45]  Andrew Maimone,et al.  Holographic optics for thin and lightweight virtual reality , 2020, ACM Trans. Graph..

[46]  Florian Müller,et al.  You Invaded my Tracking Space! Using Augmented Virtuality for Spotting Passersby in Room-Scale Virtual Reality , 2019, Conference on Designing Interactive Systems.

[47]  Per Ola Kristensson,et al.  The Office of the Future: Virtual, Portable, and Global , 2018, IEEE Computer Graphics and Applications.

[48]  Henry Been-Lirn Duh,et al.  Mixed Reality Office System Based on Maslow’s Hierarchy of Needs: Towards the Long-Term Immersion in Virtual Environments , 2019, 2019 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[49]  Tim Menzner,et al.  A Capacitive-sensing Physical Keyboard for VR Text Entry , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[50]  Jaesung Lee,et al.  A novel method for VR sickness reduction based on dynamic field of view processing , 2020, Virtual Reality.

[51]  Holger Regenbrecht,et al.  The Experience of Presence: Factor Analytic Insights , 2001, Presence: Teleoperators & Virtual Environments.

[52]  Samuel B. Williams,et al.  ASSOCIATION FOR COMPUTING MACHINERY , 2000 .

[53]  Gregory D. Abowd,et al.  NotifiVR: Exploring Interruptions and Notifications in Virtual Reality , 2018, IEEE Transactions on Visualization and Computer Graphics.

[54]  Kazuki Takashima,et al.  Text Typing in VR Using Smartphones Touchscreen and HMD , 2019, 2019 IEEE Conference on Virtual Reality and 3D User Interfaces (VR).

[55]  Tim Claudius Stratmann,et al.  PeriMR: a prototyping tool for head-mounted peripheral light displays in mixed reality , 2017, MobileHCI.

[56]  Ismo Rakkolainen,et al.  Field-of-view extension for VR viewers , 2017, MindTrek.

[57]  Wataru Yamada,et al.  Transparent Mask: Face-Capturing Head-Mounted Display with IR Pass Filters , 2018, UIST.

[58]  Daniel Pohl,et al.  See what I see: Concepts to improve the social acceptance of HMDs , 2016, 2016 IEEE Virtual Reality (VR).

[59]  Roderick Murray-Smith,et al.  A Dose of Reality: Overcoming Usability Challenges in VR Head-Mounted Displays , 2015, CHI Extended Abstracts.

[60]  Majken Kirkegaard Rasmussen,et al.  Shape-changing interfaces: a review of the design space and open research questions , 2012, CHI.

[61]  Justus Thies,et al.  FaceVR , 2018, ACM Trans. Graph..

[62]  Philip T. Kortum,et al.  Determining what individual SUS scores mean: adding an adjective rating scale , 2009 .

[63]  Mel Slater,et al.  The impact of eye gaze on communication using humanoid avatars , 2001, CHI.

[64]  Harpreet Sareen,et al.  FaceDisplay: Towards Asymmetric Multi-User Interaction for Nomadic Virtual Reality , 2018, CHI.

[65]  Wataru Yamada,et al.  Expanding the Field-of-View of Head-Mounted Displays with Peripheral Blurred Images , 2016, UIST.

[66]  Henry Fuchs,et al.  Optical Versus Video See-Through Head-Mounted Displays in Medical Visualization , 2000, Presence: Teleoperators & Virtual Environments.

[67]  Julie R. Williamson,et al.  Reality aware VR headsets , 2020, PerDis.

[68]  Liwei Chan,et al.  ShareSpace: Facilitating Shared Use of the Physical Space by both VR Head-Mounted Display and External Users , 2018, UIST.

[69]  Rafael Kuffner dos Anjos,et al.  Promoting Reality Awareness in Virtual Reality through Proxemics , 2021, 2021 IEEE Virtual Reality and 3D User Interfaces (VR).

[70]  Daniel Freedman,et al.  Reality Skins: Creating Immersive and Tactile Virtual Environments , 2016, 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[71]  Aidan Kehoe,et al.  Using Traditional Keyboards in VR: SteamVR Developer Kit and Pilot Game User Study , 2018, 2018 IEEE Games, Entertainment, Media Conference (GEM).

[72]  Gerard Jounghyun Kim,et al.  HoVR-Type: Smartphone as a typing interface in VR using hovering , 2016, 2017 IEEE International Conference on Consumer Electronics (ICCE).

[73]  Jun Rekimoto,et al.  CompoundDome: A wearable dome device that enables interaction with the real world by controlling the transparency of the screen , 2019, AH.

[74]  Kouta Minamizawa,et al.  FrontFace: facilitating communication between HMD users and outsiders using front-facing-screen HMDs , 2017, MobileHCI.