High resolution free-view interpolation of planar structure

In many applications, users desire to access a random viewing angle of a scene in high resolution while this specifically queried imagery is not among the acquired sample images. In theory, 3D reconstruction based rendering can generate such an image. However, accurate camera calibration over large scale photo collections is needed and is highly complex in nature. Image stitching based approachescan also be applied. However, such scheme is unable to provide free view interpolation or resolution enhancement. In this paper, we present a novel free view image super-resolution scheme to interpolate free views for planar structures. We construct a Bayesian model and marginalize it over photometric regulation and geometric registration parameters applying the Lie group theory. Experimental results show that the proposed scheme is able to achieve desired performance against the state-of-the-art image super-resolution approaches and successfully obtain registration in full 6 degree-of-freedom (DOF). Compared with existing image based rendering schemes, the proposed scheme achieves free view interpolation for planar structures with higher resolution and less distortion.

[1]  Stephen J. Roberts,et al.  Bayesian Image Super-resolution, Continued , 2006, NIPS.

[2]  Steven M. Seitz,et al.  Finding paths through the world's photos , 2008, SIGGRAPH 2008.

[3]  Christopher M. Bishop,et al.  Bayesian Image Super-Resolution , 2002, NIPS.

[4]  Michael Werman,et al.  How to put probabilities on homographies , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Jan-Michael Frahm,et al.  Efficient Generation of Multi-perspective Panoramas , 2011, 2011 International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission.

[6]  Steven M. Seitz,et al.  Photo tourism: exploring photo collections in 3D , 2006, ACM Trans. Graph..

[7]  Steven M. Seitz,et al.  Multicore bundle adjustment , 2011, CVPR 2011.

[8]  Nirmal K. Bose,et al.  Recursive reconstruction of high resolution image from noisy undersampled multiframes , 1990, IEEE Trans. Acoust. Speech Signal Process..

[9]  R. Szeliski.  Image Alignment and Stitching : A Tutorial 1 , 2004 .

[10]  Shmuel Peleg,et al.  Minimal Aspect Distortion (MAD) Mosaicing of Long Scenes , 2008, International Journal of Computer Vision.

[11]  Klamer Schutte,et al.  Robust Fusion of Irregularly Sampled Data Using Adaptive Normalized Convolution , 2006, EURASIP J. Adv. Signal Process..

[12]  N. K. Bose,et al.  High resolution image formation from low resolution frames using Delaunay triangulation , 2002, IEEE Trans. Image Process..

[13]  Jorge Nocedal,et al.  Algorithm 778: L-BFGS-B: Fortran subroutines for large-scale bound-constrained optimization , 1997, TOMS.

[14]  David Salesin,et al.  Photographing long scenes with multi-viewpoint panoramas , 2006, SIGGRAPH 2006.

[15]  Kyoung Mu Lee,et al.  Monocular SLAM with locally planar landmarks via geometric rao-blackwellized particle filtering on Lie groups , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.