Supporting outdoor mixed reality applications for architecture and cultural heritage

This paper introduces new approaches to enable collaborative outdoor mixed reality design review in the architectural domain as well as outdoor mixed reality experiences in the cultural heritage domain. For this purpose we present the results of three closely related European projects, IMPROVE and CINeSPACE, which are currently succeeded by MAXIMUS, continuing the development of technologies relevant to the two domains. The paper focuses on the base technologies needed to develop usable outdoor mixed reality applications, such as marker-less optical tracking combined with sensor fusion for accurate pose estimation outdoors. Furthermore the paper presents a new visualization system developed within the project, one of the few daylight blocking head.-mounted displays available. In addition to pose estimation and visualization devices, the paper presents a VR framework adapted for the architecture and the cultural heritage domain featuring high dynamic range image acquisition combined with pre-computed radiance transfer rendering to be able to photo-realistically render urban content. For the cultural heritage domain this framework has been extended to allow accurate display and generation of multimedia content super-imposed on a real environment as well as city navigation.

[1]  Peter Shirley,et al.  Realistic ray tracing , 2000 .

[2]  André Stork,et al.  The daylight blocking optical stereo see-through HMD , 2008, IPT/EDT '08.

[3]  Martin Bauer,et al.  Fata Morgana - a presentation system for product design , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[4]  Mark Billinghurst,et al.  An occlusion capable optical see-through head mount display for supporting co-located collaboration , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[5]  Jurriaan D. Mulder.  Realistic occlusion effects in mirror-based co-located augmented reality systems , 2005, IEEE Proceedings. VR 2005. Virtual Reality, 2005..

[6]  Tobias Höllerer,et al.  Hybrid Feature Tracking and User Interaction for Markerless Augmented Reality , 2008, 2008 IEEE Virtual Reality Conference.

[7]  Hiroyuki Yamamoto,et al.  A Fast Initialization Method for Edge-based Registration Using an Inclination Constraint , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[8]  Jan Kautz,et al.  Precomputed radiance transfer for real-time rendering in dynamic, low-frequency lighting environments , 2002 .

[9]  Pedro Santos,et al.  An innovative daylight blocking optical stereo see-through HMD , 2008, SIGGRAPH '08.

[10]  Soh-Khim Ong,et al.  A differential GPS carrier phase technique for precision outdoor AR tracking , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[11]  Didier Stricker,et al.  Advanced tracking through efficient image processing and visual-inertial sensor fusion , 2008, VR.

[12]  Dieter Schmalstieg,et al.  Augmented Reality Scouting for Interactive 3D Reconstruction , 2007, 2007 IEEE Virtual Reality Conference.

[13]  Stephen DiVerdi,et al.  GroundCam: A Tracking Modality for Mobile Mixed Reality , 2007, 2007 IEEE Virtual Reality Conference.

[14]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[15]  Paul Debevec.  Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography , 2008, SIGGRAPH Classes.

[16]  Pedro Santos,et al.  DISPLAY AND RENDERING TECHNOLOGIES FOR VIRTUAL AND MIXED REALITY DESIGN REVIEW , 2007 .

[17]  André Stork,et al.  Rendering techniques for mixed reality , 2010, Journal of Real-Time Image Processing.

[18]  Philipp Slusallek,et al.  Sttreaming Video Textures for Mixed Reality Applications in Interactive Ray Tracing Environments , 2003, VMV.

[19]  Tom Drummond,et al.  Initialisation for Visual Tracking in Urban Environments , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[20]  Ivan E. Sutherland,et al.  A head-mounted three dimensional display , 1968, AFIPS Fall Joint Computing Conference.

[21]  Gudrun Klinker,et al.  A System Architecture for Ubiquitous Tracking Environments , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[22]  Zhengyou Zhang,et al.  Flexible camera calibration by viewing a plane from unknown orientations , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[23]  Didier Stricker,et al.  Online camera pose estimation in partially known and dynamic scenes , 2006, 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality.

[24]  Gudrun Klinker,et al.  Dynamic gyroscope fusion in Ubiquitous Tracking environments , 2008, 2008 7th IEEE/ACM International Symposium on Mixed and Augmented Reality.

[25]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.