Mobile devices for interaction in immersive virtual environments

Gamepads and 3D controllers are the main controllers used in most Virtual Environments. Despite being simple to use, these input devices have a number of limitations as fixed layout and difficulty to remember the mapping between buttons and functions. Mobile devices present interesting characteristics that might be valuable in immersive environments: more flexible interfaces, touchscreen combined with onboard sensors that allow new interaction and easy acceptance since these devices are used daily by most users. The work described in this article proposes a solution that uses mobile devices to interact with Immersive Virtual Environments for selection and navigation tasks. The proposed solution uses the mobile device camera to track the Head-Mounted-Display position and present a virtual representation of the mobile device screen; it was tested using an Immersive Virtual Museum as use case. Based on this prototype, a study was performed to compare controller based and mobile based interaction for navigation and selection showing that using mobile devices is viable in this context and offers interesting interaction opportunities.

[1]  Christoph W. Borst,et al.  Handymenu: Integrating menu selection into a multifunction smartphone-based VR controller , 2015, 2015 IEEE Symposium on 3D User Interfaces (3DUI).

[2]  Borko Furht,et al.  Encyclopedia of Multimedia , 2006 .

[3]  A. Raposo,et al.  A tablet-based 3D interaction tool for virtual engineering environments , 2013, VRCAI '13.

[4]  Klaus Jung,et al.  The smartphone as a 3D input device , 2012, 2012 IEEE Second International Conference on Consumer Electronics - Berlin (ICCE-Berlin).

[5]  Mark T. Bolas,et al.  Tablet-based interaction panels for immersive environments , 2014, 2014 IEEE Virtual Reality (VR).

[6]  Marc Erich Latoschik,et al.  Avatar anthropomorphism and illusion of body ownership in VR , 2015, 2015 IEEE Virtual Reality (VR).

[7]  Sebastian Thelen,et al.  Using smart phones for large-display interaction , 2011, 2011 International Conference on User Science and Engineering (i-USEr ).

[8]  M. Raynal,et al.  Smartphone-Based 3 D Navigation Technique for Use in a Museum Exhibit , 2014 .

[9]  Benjamin Lok,et al.  Experiences in Using a Smartphone as a Virtual Reality Interaction Device , 2012 .

[10]  Achim Ebert,et al.  ModControl - Mobile Phones as a Versatile Interaction Device for Large Screen Applications , 2011, INTERACT.

[11]  Samuel B. Williams,et al.  ASSOCIATION FOR COMPUTING MACHINERY , 2000 .

[12]  Victoria Interrante,et al.  The effect of self-embodiment on distance perception in immersive virtual environments , 2008, VRST '08.

[13]  Paulo Dias,et al.  Effect of hand-avatar in a selection task using a tablet as input device in an immersive virtual environment , 2017, 2017 IEEE Symposium on 3D User Interfaces (3DUI).

[14]  Paulo Dias,et al.  Gesture Interactions for Virtual Immersive Environments: Navigation, Selection and Manipulation , 2016, HCI.

[15]  Tuukka M. Takala,et al.  Full body interaction in virtual reality with affordable hardware , 2014, 2014 IEEE Virtual Reality (VR).

[16]  Borivoje Furht.  Encyclopedia of multimedia : A-Z , 2008 .

[17]  Ivan Poupyrev,et al.  3D User Interfaces: Theory and Practice , 2004 .

[18]  Perttu Hämäläinen,et al.  Immersive 3D modeling with Blender and off-the-shelf hardware , 2013, 2013 IEEE Symposium on 3D User Interfaces (3DUI).

[19]  Masahiro Hori,et al.  Mobile devices as multi-DOF controllers , 2010, 2010 IEEE Symposium on 3D User Interfaces (3DUI).

[20]  Regis Kopper,et al.  Comparative study of input devices for a VR mine simulation , 2014, 2014 IEEE Virtual Reality (VR).

[21]  Emiko Charbonneau,et al.  The Wiimote and Beyond: Spatially Convenient Devices for 3D User Interfaces , 2010, IEEE Computer Graphics and Applications.

[22]  Simon J. Julier,et al.  Design and implementation of an immersive virtual reality system based on a smartphone platform , 2013, 2013 IEEE Symposium on 3D User Interfaces (3DUI).

[23]  Gudrun Klinker,et al.  Phone-based motion control in VR: analysis of degrees of freedom , 2011, CHI EA '11.

[24]  Frank Biocca,et al.  Connected to My Avatar: - Effects of Avatar Embodiments on User Cognitions, Behaviors, and Self Construal , 2014, HCI.

[25]  Paulo Dias,et al.  Platform for setting up interactive virtual environments , 2014, Electronic Imaging.

[26]  Colin Ware,et al.  Exploration and virtual camera control in virtual three dimensional environments , 1990, I3D '90.

[27]  Anderson Maciel,et al.  Collaborative 3D manipulation using mobile phones , 2016, 2016 IEEE Symposium on 3D User Interfaces (3DUI).

[28]  Daniel Medeiros,et al.  Proposal and evaluation of a tablet-based tool for 3D virtual environments , 2014 .