Occlusion registration in video-based augmented reality

Augmented Reality overlaps virtual objects on real world. It mixes a real and virtual world which can generate more semantic meanings than either one. To seamlessly merge virtual and real objects, model registration is an important problem. This paper provides a practical approach to do occlusion registration which is a key cue for users to understand the scene. We apply our method to Video-based Augmented Reality where detecting occlusion relationship is challenging because virtual objects are simply superimposed on images of real scenes. By estimating the dense depth of real objects from stereo, results show our approach can efficiently and correctly register virtual and real objects.

[1]  Peter Meer,et al.  Estimation of Nonlinear Errors-in-Variables Models for Computer Vision Applications , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Olivier Bernier,et al.  Local kernel color histograms for background subtraction , 2006, VISAPP.

[3]  Andrew Y. C. Nee,et al.  A markerless registration method for augmented reality based on affine properties , 2006, AUIC.

[4]  Yuichi Ohta,et al.  Share-Z: Client/Server Depth Sensing for See-Through Head-Mounted Displays , 2002, Presence: Teleoperators & Virtual Environments.

[5]  Patrick Hébert,et al.  Handling Occlusions in Real-time Augmented Reality : Dealing with Movable Real and Virtual Objects , 2006, The 3rd Canadian Conference on Computer and Robot Vision (CRV'06).

[6]  Michael Gervautz,et al.  Occlusion in collaborative augmented environments , 1999, Comput. Graph..

[7]  Ryutarou Ohbuchi,et al.  Merging virtual objects with the real world: seeing ultrasound imagery within the patient , 1992, SIGGRAPH.

[8]  Kwanghoon Sohn,et al.  3D reconstruction of stereo images for interaction between real and virtual worlds , 2003, The Second IEEE and ACM International Symposium on Mixed and Augmented Reality, 2003. Proceedings..

[9]  Mark Billinghurst,et al.  The use of dense stereo range data in augmented reality , 2002, Proceedings. International Symposium on Mixed and Augmented Reality.

[10]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[11]  Naokazu Yokoya,et al.  Stereo Vision Based Video See-through Mixed Reality , 1999 .

[12]  Vincent Lepetit,et al.  A semi-automatic method for resolving occlusion in augmented reality , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[13]  Shogo Nishida,et al.  Occlusion detection of real objects using contour based stereo matching , 2005, ICAT '05.

[14]  Ramesh Raskar,et al.  Modern approaches to augmented reality: introduction to current approaches , 2006, SIGGRAPH Courses.

[15]  Deborah Hix,et al.  A Perceptual Matching Technique for Depth Judgments in Optical, See-Through Augmented Reality , 2006, IEEE Virtual Reality Conference (VR 2006).