Mixed Voxel Reality: Presence and Embodiment in Low Fidelity, Visually Coherent, Mixed Reality Environments

Mixed Reality aims at combining virtual reality with the user's surrounding real environment in a way that they form one, coherent reality. A coherent visual quality is of utmost importance, expressed in measures of e.g. resolution, framerate, and latency for both the real and the virtual domains. For years, researchers have focused on maximizing the quality of the virtual visualization mimicking the real world to get closer to visual coherence. This however, makes Mixed Reality systems overly complex and requires high computational power. In this paper, we propose a different approach by decreasing the realism of one or both visual realms, real and virtual, to achieve visual coherence. Our system coarsely voxelizes the real and virtual environments, objects, and people to provide a believable, coherent mixed voxel reality. In this paper we present the general idea, the current implementation and demonstrate the effectiveness of our approach by technical and empirical evaluations. Our mixed voxel reality system serves as a platform for low-cost presence research and studies on human perception and cognition, a host of diagnostic and therapeutic applications, and for a variety of Mixed Reality applications where users' embodiment is important. Our findings challenge some commonplace assumptions on more is better approaches in mixed reality research and practice—sometimes less can be more.

[1]  Anselmo Lastra,et al.  From Motion to Photons in 80 Microseconds: Towards Minimal Latency for Virtual and Augmented Reality , 2016, IEEE Transactions on Visualization and Computer Graphics.

[2]  Marc Erich Latoschik,et al.  Avatar embodiment realism and virtual fitness training , 2015, 2015 IEEE Virtual Reality (VR).

[3]  Mel Slater,et al.  Visual Realism Enhances Realistic Response in an Immersive Virtual Environment , 2009, IEEE Computer Graphics and Applications.

[4]  Bernd Fröhlich,et al.  Immersive Group-to-Group Telepresence , 2013, IEEE Transactions on Visualization and Computer Graphics.

[5]  Arindam Dey,et al.  BurnAR: Feel the heat , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[6]  Holger Regenbrecht,et al.  Mixed Reality Experience Questionnaire (MREQ) -Reference , 2017 .

[7]  Pedro Arias,et al.  Metrological comparison between Kinect I and Kinect II sensors , 2015 .

[8]  Dieter Schmalstieg,et al.  OmniKinect: real-time dense volumetric data acquisition and applications , 2012, VRST '12.

[9]  Céline Loscos,et al.  Classification of Illumination Methods for Mixed Reality , 2006, Comput. Graph. Forum.

[10]  Steven K. Feiner,et al.  WeARHand: Head-worn, RGB-D camera-based, bare-hand user interface with visually enhanced depth perception , 2014, 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[11]  M. Slater,et al.  Illusory ownership of a virtual child body causes overestimation of object sizes and implicit attitude changes , 2013, Proceedings of the National Academy of Sciences.

[12]  Robert S. Kennedy,et al.  Simulator Sickness Questionnaire: An enhanced method for quantifying simulator sickness. , 1993 .

[13]  Chris Shaw,et al.  On temporal-spatial realism in the virtual reality environment , 1991, UIST '91.

[14]  Anthony Steed A simple method for estimating the latency of interactive, real-time graphics simulations , 2008, VRST '08.

[15]  Philipp Lensing,et al.  Achieving Flexible 3D Reconstruction Volumes for RGB-D and RGB Camera Based Approaches , 2016, ICCVG.

[16]  Mel Slater,et al.  How to Build an Embodiment Lab: Achieving Body Representation Illusions in Virtual Reality , 2014, Front. Robot. AI.

[17]  Svenja Kahn,et al.  Reducing the gap between Augmented Reality and 3D modeling with real-time depth imaging , 2013, Virtual Reality.

[18]  Hirokazu Kato,et al.  The effects of shadow representation of virtual objects in augmented reality , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[19]  Anselm Grundhöfer,et al.  A LED-Based IR/RGB End-to-End Latency Measurement Device , 2016, 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR-Adjunct).

[20]  Holger Regenbrecht,et al.  Manipulating the Experience of Reality for Rehabilitation Applications , 2014, Proceedings of the IEEE.

[21]  Kamel Hamrouni,et al.  Noise Modelling and Uncertainty Propagation for TOF Sensors , 2012, ECCV Workshops.

[22]  David W. Murray,et al.  Compositing for small cameras , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[23]  Kay M. Stanney,et al.  Configural Scoring of Simulator Sickness, Cybersickness and Space Adaptation Syndrome: Similarities and Differences? , 2001 .

[24]  Karl F. MacDorman,et al.  Measuring the Uncanny Valley Effect , 2017, Int. J. Soc. Robotics.

[25]  Mark A. Livingston,et al.  Managing latency in complex augmented reality systems , 1997, SI3D.

[26]  P. Milgram,et al.  A Taxonomy of Mixed Reality Visual Displays , 1994 .

[27]  Remo Sala,et al.  A metrological characterization of the Kinect V2 time-of-flight camera , 2016, Robotics Auton. Syst..

[28]  Wei Xie,et al.  Garden: A Mixed Reality Experience Combining Virtual Reality and 3D Reconstruction , 2016, CHI Extended Abstracts.

[29]  Colin Swindells,et al.  System lag tests for augmented and virtual environments , 2000, UIST '00.

[30]  Joan Llobera,et al.  The relationship between virtual body ownership and temperature sensitivity , 2013, Journal of The Royal Society Interface.

[31]  Maria V. Sanchez-Vives,et al.  Inducing Illusory Ownership of a Virtual Body , 2009, Front. Neurosci..

[32]  Gerhard Reitmayr,et al.  Image-based clothes transfer , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[33]  Jan-Michael Frahm,et al.  Scanning and tracking dynamic objects with commodity depth cameras , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[34]  Mel Slater,et al.  The Sense of Embodiment in Virtual Reality , 2012, PRESENCE: Teleoperators and Virtual Environments.

[35]  Marc Olano,et al.  Combatting rendering latency , 1995, I3D '95.

[36]  Holger Regenbrecht,et al.  Real and Illusory Interactions Enhance Presence in Virtual Environments , 2002, Presence: Teleoperators & Virtual Environments.

[37]  Charles T. Loop,et al.  Holoportation: Virtual 3D Teleportation in Real-time , 2016, UIST.

[38]  Nassir Navab,et al.  Measurement of absolute latency for video see through augmented reality , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[39]  Holger Regenbrecht,et al.  The Experience of Presence: Factor Analytic Insights , 2001, Presence: Teleoperators & Virtual Environments.

[40]  Horst Bischof,et al.  A Free-Viewpoint Virtual Mirror with Marker-Less User Interaction , 2011, SCIA.

[41]  Dirk Bartz,et al.  Stylized augmented reality for improved immersion , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..