Realistic occlusion effects in mirror-based co-located augmented reality systems

This paper describes the incorporation of realistic occlusion effects into mirror-based, stereoscopic co-location augmented reality display systems. By adding a light-blocking device in the form of an LCD panel underneath the semi-transparent mirror, the view on the physical world can be selectively blocked out such that virtual objects can fully occlude physical objects. Furthermore, by removing areas of the virtual objects rendered on the reflected display, physical objects seen through the semi-transparent mirror and the transmissive LCD panel appear to occlude these virtual objects. We describe the governing principles of the approach, and present an efficient algorithm for the generation of the occlusion masks with the use of vision-based scene reconstruction. Finally, a first prototype implementation of the system is presented.

[1]  Christopher Schmandt,et al.  Spatial input/display correspondence in a stereoscopic computer graphic work station , 1983, SIGGRAPH.

[2]  Colin Ware,et al.  Eye-hand co-ordination with force feedback , 2000, CHI.

[3]  Hiroyuki Ohno,et al.  An optical see-through display for mutual occlusion of real and virtual environments , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[4]  Luis Serra,et al.  Interactive vessel tracing in volume data , 1997, SI3D.

[5]  Benjamin Lok,et al.  Online model reconstruction for interactive virtual environments , 2001, I3D '01.

[6]  Hans-Peter Seidel,et al.  Hardware-Accelerated Visual Hull Reconstruction and Rendering , 2003, Graphics Interface.

[7]  Mark Billinghurst,et al.  An occlusion capable optical see-through head mount display for supporting co-located collaboration , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[8]  Robert van Liere,et al.  Optical tracking using projective invariant marker pattern properties , 2003, IEEE Virtual Reality, 2003. Proceedings..

[9]  David W. Schloerb,et al.  Virtual Workbench: Near-Field Virtual Environment System with Applications , 1999, Presence.

[10]  Mary C. Whitton,et al.  Effects of handling real objects and avatar fidelity on cognitive task performance in virtual environments , 2003, IEEE Virtual Reality, 2003. Proceedings..

[11]  Wojciech Matusik,et al.  Polyhedral Visual Hulls for Real-Time Rendering , 2001, Rendering Techniques.

[12]  Ramesh Raskar,et al.  Image-based visual hulls , 2000, SIGGRAPH.

[13]  Jurriaan D. Mulder,et al.  A modular system for collaborative desktop VR/AR with a shared workspace , 2004, IEEE Virtual Reality 2004.

[14]  Andrew E. Johnson,et al.  Developing the PARIS : Using the CAVE to Prototype a New VR Display , 2000 .

[15]  Andrea H. Mason,et al.  Reaching movements to augmented and graphic objects in virtual environments , 2001, CHI.

[16]  Oliver Bimber,et al.  Occlusion shadows: using projected light to generate realistic occlusion effects for view-dependent optical see-through displays , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[17]  Helmut Seibert,et al.  A flexible tracking concept applied to medical scenarios using an AR window , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[18]  J. C. Mulder,et al.  The personal space station: Bringing interaction within reach , 2002 .

[19]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Edmond Boyer,et al.  Real-Time Capture, Reconstruction and Insertion into Virtual World of Human Actors , 2003, VVG.

[21]  Michael Gervautz,et al.  Occlusion in collaborative augmented environments , 1999, Comput. Graph..

[22]  Jeff Rose,et al.  Rotating virtual objects with real handles , 1999, TCHI.

[23]  Greg Welch,et al.  The office of the future: a unified approach to image-based modeling and spatially immersive displays , 1998, SIGGRAPH.

[24]  Jurriaan D. Mulder,et al.  An affordable optical head tracking system for desktop VR/AR systems , 2003 .

[25]  Luis Serra,et al.  Dextrous virtual work , 1996, CACM.

[26]  Ruigang Yang,et al.  Real‐Time Consensus‐Based Scene Reconstruction Using Commodity Graphics Hardware † , 2003, Comput. Graph. Forum.