Occlusion handling using moving volume and ray casting techniques for augmented reality systems

Obtaining the correct occlusion relationship between real objects and virtual objects is vital for improving augmented reality technology. In this paper, we propose a novel occlusion handling method using moving volume and ray casting techniques. Our method is divided into two steps. In the first step, we obtain the volume of the corresponding physical space and arbitrarily move the volume to extend the reconstruction area. In the second step, we calculate the 3D coordinates of each pixel in the scene and re-project the rendered objects to the same 3D coordinates system. Correct occlusion relationships are obtained by comparing the z coordinates of real and virtual objects. Several experiments are performed to validate the performance of the proposed method. The experimental results indicate that our method can correctly and rapidly handle occlusion.

[1]  Yen-Lin Chen,et al.  Edge Snapping-Based Depth Enhancement for Dynamic Occlusion Handling in Augmented Reality , 2016, 2016 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[2]  David E. Breen,et al.  Interactive Occlusion and Automatic Object Placement for Augmented Reality , 1996, Comput. Graph. Forum.

[3]  Liya Duan,et al.  Wide area registration on camera phones for mobile augmented reality applications , 2013 .

[4]  Andrew W. Fitzgibbon,et al.  KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera , 2011, UIST.

[5]  Marsette Vona,et al.  Moving Volume KinectFusion , 2012, BMVC.

[6]  Heinrich Niemann,et al.  Dense disparity maps in real-time with an application to augmented reality , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[7]  Rongrong Ji,et al.  On-Device Mobile Landmark Recognition Using Binarized Descriptor with Multifeature Fusion , 2015, ACM Trans. Intell. Syst. Technol..

[8]  Nassir Navab,et al.  Real-Time 3D Reconstruction for Occlusion-Aware Interactions in Mixed Reality , 2009, ISVC.

[9]  Dirk Bartz,et al.  Occlusion handling for medical augmented reality using a volumetric phantom model , 2004, VRST '04.

[10]  Junqing Yu,et al.  Dense 3D reconstruction combining depth and RGB information , 2016, Neurocomputing.

[11]  Suyang Dong,et al.  Visual Occlusion in Outdoor Augmented Reality Using TOF Camera and OpenGL Frame Buffer , 2010 .

[12]  Slobodan Ilic,et al.  RGB-D camera-based parallel tracking and meshing , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[13]  Jiejie Zhu,et al.  Handling occlusions in video‐based augmented reality using depth information , 2010, Comput. Animat. Virtual Worlds.

[14]  Michael Gervautz,et al.  Occlusion in collaborative augmented environments , 1999, Comput. Graph..

[15]  Vincent Lepetit,et al.  Handling occlusion in augmented reality systems: a semi-automatic method , 2000, Proceedings IEEE and ACM International Symposium on Augmented Reality (ISAR 2000).

[16]  Vincent Lepetit,et al.  A semi-automatic method for resolving occlusion in augmented reality , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[17]  Cheng Wang,et al.  An automatic occlusion handling method in augmented reality , 2010 .

[18]  Takeshi Oishi,et al.  Reduction of contradictory partial occlusion in mixed reality by using characteristics of transparency perception , 2012, 2012 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[19]  Tetsuya Kakuta,et al.  Foreground and shadow occlusion handling for outdoor augmented reality , 2010, 2010 IEEE International Symposium on Mixed and Augmented Reality.

[20]  Tiow Seng Tan,et al.  Resolving occlusion in image sequence made easy , 1998, The Visual Computer.

[21]  Romero Tori,et al.  Mutual occlusion between real and virtual elements in Augmented Reality based on fiducial markers , 2012, 2012 IEEE Workshop on the Applications of Computer Vision (WACV).

[22]  Junqing Yu,et al.  Wide area localization and tracking on camera phones for mobile augmented reality systems , 2015, Multimedia Systems.