Achieving robust alignment for outdoor mixed reality using 3D range data

Mixed reality (MR) technology can be applied to various applications such as architecture, advertising, and navigation systems, so the desire to utilize MR in outdoor environments has been increasing. In order to utilize MR, it is necessary to achieve alignment super imposing virtual contents in the desired position. However, because light changes continually in outdoor environments, and the appearance of real objects changes also, in some cases the previous image-based alignment methods do not work well. In this paper, a robust image-based alignment method to be used in outdoor environments is proposed. In the proposed method, the albedo of real objects is estimated using 3D shapes of these objects in advance, and the appearance is reproduced from the albedo and current light environment. The appearance of real objects and reproduced image becomes close, so a robust image-based alignment is achieved.

[1]  Katsushi Ikeuchi,et al.  Simple Surface Reflectance Estimation of Diffuse Outdoor Object using Spherical Images , 2007 .

[2]  T. Drummond,et al.  Going out : Robust Tracking for Outdoor Augmented Reality , 2006 .

[3]  George Papagiannakis,et al.  Mixing virtual and real scenes in the site of ancient Pompeii: Research Articles , 2005 .

[4]  George Papagiannakis,et al.  Mixing virtual and real scenes in the site of ancient Pompeii , 2005, Comput. Animat. Virtual Worlds.

[5]  Ronald Azuma,et al.  A Survey of Augmented Reality , 1997, Presence: Teleoperators & Virtual Environments.

[6]  Dieter Schmalstieg,et al.  Real-time panoramic mapping and tracking on mobile phones , 2010, 2010 IEEE Virtual Reality Conference (VR).

[7]  S. Müller,et al.  Analysis by Synthesis Techniques for Markerless Tracking , 2012 .

[8]  Aly A. Farag,et al.  CSIFT: A SIFT Descriptor with Color Invariant Characteristics , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[9]  Atsushi Nakazawa,et al.  Parallel alignment of a large number of range images , 2003 .

[10]  Paul E. Debevec,et al.  Image-based lighting , 2002, IEEE Computer Graphics and Applications.

[11]  Ronald Azuma,et al.  Recent Advances in Augmented Reality , 2001, IEEE Computer Graphics and Applications.

[12]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[13]  Atsushi Nakazawa,et al.  Parallel alignment of a large number of range images , 2003, Fourth International Conference on 3-D Digital Imaging and Modeling, 2003. 3DIM 2003. Proceedings..

[14]  Paul R. Cohen,et al.  Camera Calibration with Distortion Models and Accuracy Evaluation , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Didier Stricker,et al.  Adaptable Model-Based Tracking Using Analysis-by-Synthesis Techniques , 2007, CAIP.

[16]  Dieter Schmalstieg,et al.  Global pose estimation using multi-sensor fusion for outdoor Augmented Reality , 2009, 2009 8th IEEE International Symposium on Mixed and Augmented Reality.

[17]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[18]  Gilles Simon,et al.  Tracking-by-synthesis using point features and pyramidal blurring , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[19]  Naokazu Yokoya,et al.  Real-time geometric registration using feature landmark database for augmented reality applications , 2009, Electronic Imaging.

[20]  Stephen DiVerdi,et al.  Envisor: Online Environment Map Construction for Mixed Reality , 2008, 2008 IEEE Virtual Reality Conference.

[21]  Tom Drummond,et al.  Machine Learning for High-Speed Corner Detection , 2006, ECCV.

[22]  Tetsuya Kakuta,et al.  Fast Shading and Shadowing of Virtual Objects Using Shadowing Planes in Mixed Reality , 2008 .

[23]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[24]  G. Klein,et al.  Parallel Tracking and Mapping for Small AR Workspaces , 2007, 2007 6th IEEE and ACM International Symposium on Mixed and Augmented Reality.

[25]  M. H. Brill,et al.  Heuristic analysis of von Kries color constancy. , 1986, Journal of the Optical Society of America. A, Optics and image science.

[26]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[28]  Didier Stricker,et al.  Advanced tracking through efficient image processing and visual-inertial sensor fusion , 2008, 2008 IEEE Virtual Reality Conference.

[29]  Atsushi Nakazawa,et al.  The great buddha project: modeling cultural heritage for VR systems through observation , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[30]  ZhangZhengyou.  A Flexible New Technique for Camera Calibration , 2000 .

[31]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[32]  Tom Drummond,et al.  Going out: robust model-based tracking for outdoor augmented reality , 2006, 2006 IEEE/ACM International Symposium on Mixed and Augmented Reality.

[33]  Vincent Lepetit,et al.  Fully automated and stable registration for augmented reality applications , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[34]  Andrew J. Davison,et al.  DTAM: Dense tracking and mapping in real-time , 2011, 2011 International Conference on Computer Vision.

[35]  Didier Stricker,et al.  Archeoguide: An Augmented Reality Guide for Archaeological Sites , 2002, IEEE Computer Graphics and Applications.