Diminished Reality for Landscape Video Sequences with Homographies

This paper describes a Diminished Reality application to eliminate occluders on videos capturing distant landscapes. Although many tourist spots have lookouts to see the distant landscape, the video capturing from there usually includes obstructive occluders such as pillar and muntin. The application vanishes the occluders in the video in semi-interactively with homographies of distant landscape among frames. For this purpose, the regions of occluders in every frame are segmented by the GrabCut algorism, and then the each region is recovered by the appropriate texture from another frame which is not occluded in same region.

[1]  David G. Lowe,et al.  Object recognition from local scale-invariant features , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[2]  Vincent Lepetit,et al.  A semi-automatic method for resolving occlusion in augmented reality , 2000, Proceedings IEEE Conference on Computer Vision and Pattern Recognition. CVPR 2000 (Cat. No.PR00662).

[3]  Irfan A. Essa,et al.  Graphcut textures: image and video synthesis using graph cuts , 2003, ACM Trans. Graph..

[4]  Guillermo Sapiro,et al.  Image inpainting , 2000, SIGGRAPH.

[5]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Tony F. Chan,et al.  Nontexture Inpainting by Curvature-Driven Diffusions , 2001, J. Vis. Commun. Image Represent..

[7]  Marc Levoy,et al.  Fast texture synthesis using tree-structured vector quantization , 2000, SIGGRAPH.

[8]  Jiaya Jia,et al.  Video repairing: inference of foreground and background under severe occlusion , 2004, CVPR 2004.

[9]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[10]  Patrick Pérez,et al.  Region filling and object removal by exemplar-based image inpainting , 2004, IEEE Transactions on Image Processing.

[11]  Patrick Pérez,et al.  Joint Tracking and Segmentation of Objects Using Graph Cuts , 2007, ACIVS.

[12]  Eli Shechtman,et al.  Space-time video completion , 2004, CVPR 2004.

[13]  Patrick Pérez,et al.  Object removal by exemplar-based inpainting , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..