Augmented Mirrors

A recurrent problem in egocentric Augmented Reality (AR) applications is the misestimation of depth. Providing alternative views from non-egocentric perspectives can convey useful information for applications that require the correct judgment of depth as it is in the case of placement and alignment of virtual and real content, but also for exploration and visualization tasks.In this paper, we introduce Augmented Mirrors. Through the integration of a real mirror, our approach is capable to reflect changes of the real and virtual content of an AR application while users benefit from the perceptual advantages of using mirrors. Our concept, simple yet effective, only requires tracking the user and mirror poses with the accuracy demanded by a specific application. To showcase the potential and flexibility of the Augmented Mirrors, we present and discuss multiple examples ranging from alignment, exploration, spatial understanding, and selective content visualization using different AR-enabled devices and tracking technologies. We envision the Augmented Mirrors as a new and valuable concept that can be used in applications that benefit from additional viewpoints and require the simultaneous visualization of real and virtual content.

[1]  Eric Kolstad,et al.  Egocentric depth judgments in optical, see-through augmented reality , 2007, IEEE Transactions on Visualization and Computer Graphics.

[2]  Peter Willemsen,et al.  Does the Quality of the Computer Graphics Matter when Judging Distances in Visually Immersive Environments? , 2004, Presence: Teleoperators & Virtual Environments.

[3]  Colin Ware,et al.  Information Visualization: Perception for Design , 2000 .

[4]  Paul Milgram,et al.  Perceptual issues in augmented reality , 1996, Electronic Imaging.

[5]  Michael Goesele,et al.  Reconstructing scenes with mirror and glass surfaces , 2018, ACM Trans. Graph..

[6]  Peter Eisert,et al.  Virtual Mirror: Real-Time Tracking of Shoes in Augmented Reality Environments , 2007, 2007 IEEE International Conference on Image Processing.

[7]  Yuichi Ohta,et al.  MR-Mirror: A Complex of Real and Virtual Mirrors , 2009, HCI.

[8]  Nassir Navab,et al.  Laparoscopic Virtual Mirror New Interaction Paradigm for Monitor Based Augmented Reality , 2007, 2007 IEEE Virtual Reality Conference.

[9]  Tyrone L. Vincent,et al.  Analysis of Head Pose Accuracy in Augmented Reality , 2000, IEEE Trans. Vis. Comput. Graph..

[10]  Nassir Navab,et al.  The Virtual Mirror: A New Interaction Paradigm for Augmented Reality Environments , 2009, IEEE Transactions on Medical Imaging.

[11]  Mark Billinghurst,et al.  Optical-Reflection Type 3D Augmented Reality Mirrors , 2019, VRST.

[12]  Xabier E. Barandiaran,et al.  A Dynamical Systems Account of Sensorimotor Contingencies , 2013, Front. Psychol..

[13]  Peter Willemsen,et al.  The Influence of Restricted Viewing Conditions on Egocentric Distance Perception: Implications for Real and Virtual Indoor Environments , 2005, Perception.

[14]  Sergio Casas,et al.  Interacting with Augmented Reality Mirrors , 2016 .

[15]  Douglas C. Giancoli,et al.  Physics for scientists and engineers with modern physics (4th edition) , 2008 .

[16]  M. Donath,et al.  A virtual mirror for assisting drivers , 2000, Proceedings of the IEEE Intelligent Vehicles Symposium 2000 (Cat. No.00TH8511).

[17]  Jack M. Loomis,et al.  Limited Field of View of Head-Mounted Displays Is Not the Cause of Distance Underestimation in Virtual Environments , 2004, Presence: Teleoperators & Virtual Environments.

[18]  Inmaculada Coma,et al.  Augmented Mirror: Interactive Augmented Reality System Based on Kinect , 2011, INTERACT.

[19]  Mark Fiala,et al.  Magic Mirror System with Hand-held and Wearable Augmentations , 2007, 2007 IEEE Virtual Reality Conference.

[20]  Peter Willemsen,et al.  The effects of head-mounted display mechanics on distance judgments in virtual environments , 2004, APGV '04.

[21]  James E. Cutting,et al.  Chapter 3 – Perceiving Layout and Knowing Distances: The Integration, Relative Potency, and Contextual Use of Different Information about Depth* , 1995 .

[22]  Danica Kragic,et al.  Active 3D scene segmentation and detection of unknown objects , 2010, 2010 IEEE International Conference on Robotics and Automation.

[23]  Marc Erich Latoschik,et al.  FakeMi: a fake mirror system for avatar embodiment studies , 2016, VRST.

[24]  Jean Meunier,et al.  3D Reconstruction With Time-of-Flight Depth Camera and Multiple Mirrors , 2018, IEEE Access.

[25]  Dan Xia,et al.  Handling occlusions in augmented reality based on 3D reconstruction method , 2015, Neurocomputing.

[26]  Mathias Unberath,et al.  Reflective-AR Display: An Interaction Methodology for Virtual-Real Alignment in Medical Robotics , 2019, ArXiv.

[27]  Christian Geiger,et al.  Mobile augmented reality illustrations that entertain and inform: design and implementation issues with the hololens , 2017, SIGGRAPH ASIA Mobile Graphics and Interactive Applications.

[28]  Frank H. Durgin,et al.  Distance Perception and the Visual Horizon in Head-Mounted Displays , 2005, TAP.

[29]  Maud Marchal,et al.  The Mind-Mirror: See your brain in action in your head using EEG and augmented reality , 2014, 2014 IEEE Virtual Reality (VR).

[30]  Alex Pentland,et al.  The ALIVE system: wireless, full-body interaction with autonomous agents , 1997, Multimedia Systems.

[31]  Jack M. Loomis,et al.  Visual perception of egocentric distance in real and virtual environments. , 2003 .

[32]  T Paul Singh,et al.  Reverse Alignment “Mirror Image” Visualization as a Laparoscopic Training Tool Improves Task Performance , 2010, Surgical innovation.

[33]  Klaus Radermacher,et al.  Assessment of optical localizer accuracy for computer aided surgery systems , 2010, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[34]  Jae Seok Jang,et al.  Two-Phase Calibration for a Mirror Metaphor Augmented Reality System , 2014, Proceedings of the IEEE.

[35]  Tom Drummond,et al.  Sensor fusion and occlusion refinement for tablet-based AR , 2004, Third IEEE and ACM International Symposium on Mixed and Augmented Reality.

[36]  Johannes Kopf,et al.  Fast depth densification for occlusion-aware augmented reality , 2018, ACM Trans. Graph..

[37]  Nassir Navab,et al.  mirracle: An augmented reality magic mirror system for anatomy education , 2012, 2012 IEEE Virtual Reality Workshops (VRW).

[38]  Marie-Odile Berger.  Resolving occlusion in augmented reality: a contour based approach without 3D reconstruction , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[39]  Steven K. Feiner,et al.  Quick viewpoint switching for manipulating virtual objects in hand-held augmented reality using stored snapshots , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[40]  Eric Kolstad,et al.  The Effects of Virtual Reality, Augmented Reality, and Motion Parallax on Egocentric Depth Perception , 2008, 2008 IEEE Virtual Reality Conference.

[41]  David E. Breen,et al.  Interactive Occlusion and Automatic Object Placement for Augmented Reality , 1996, Comput. Graph. Forum.

[42]  Robert L Sainburg,et al.  The roles of vision and proprioception in the planning of reaching movements. , 2009, Advances in experimental medicine and biology.

[43]  Wei Liu,et al.  Pixel2Mesh: Generating 3D Mesh Models from Single RGB Images , 2018, ECCV.

[44]  Klaus Schilling,et al.  Occlusion Handling in Augmented Reality User Interfaces for Robotic Systems , 2010, ISR/ROBOTIK.