As-Projective-As-Possible Image Stitching with Moving DLT

We investigate projective estimation under model inadequacies, i.e., when the underpinning assumptions of the projective model are not fully satisfied by the data. We focus on the task of image stitching which is customarily solved by estimating a projective warp - a model that is justified when the scene is planar or when the views differ purely by rotation. Such conditions are easily violated in practice, and this yields stitching results with ghosting artefacts that necessitate the usage of deghosting algorithms. To this end we propose as-projective-as-possible warps, i.e., warps that aim to be globally projective, yet allow local non-projective deviations to account for violations to the assumed imaging conditions. Based on a novel estimation technique called Moving Direct Linear Transformation (Moving DLT), our method seamlessly bridges image regions that are inconsistent with the projective model. The result is highly accurate image stitching, with significantly reduced ghosting effects, thus lowering the dependency on post hoc deghosting.

[1]  Kyungdon Joo,et al.  Line meets as-projective-as-possible image stitching with moving DLT , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[2]  Michael S. Brown,et al.  In Defence of RANSAC for Outlier Rejection in Deformable Registration , 2012, ECCV.

[3]  Kok-Lim Low,et al.  Aligning images in the wild , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[4]  Yasuyuki Matsushita,et al.  Smoothly varying affine stitching , 2011, CVPR 2011.

[5]  Michael S. Brown,et al.  Constructing image panoramas using dual-homography warping , 2011, CVPR 2011.

[6]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[7]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[8]  Michael Gleicher,et al.  Content-preserving warps for 3D video stabilization , 2009, ACM Trans. Graph..

[9]  P. Stange.  On the efficient update of the Singular Value Decomposition , 2008 .

[10]  Chi-Keung Tang,et al.  Image Stitching Using Structure Deformation , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  M. Gross,et al.  Algebraic point set surfaces , 2007, ACM Trans. Graph..

[12]  B. Schölkopf,et al.  Non-rigid point set registration: Coherent Point Drift , 2007 .

[13]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[14]  Miguel Á. Carreira-Perpiñán,et al.  Non-rigid point set registration: Coherent Point Drift , 2006, NIPS.

[15]  Scott Schaefer,et al.  Image deformation using moving least squares , 2006, ACM Trans. Graph..

[16]  Michael F. Cohen,et al.  Photographing long scenes with multi-viewpoint panoramas , 2006, ACM Trans. Graph..

[17]  Richard Szeliski,et al.  Seamless Image Stitching of Scenes with Large Motions and Exposure Differences , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[18]  Richard Szeliski,et al.  Image Alignment and Stitching , 2006, Handbook of Mathematical Models in Computer Vision.

[19]  Nikos Paragios,et al.  Handbook of Mathematical Models in Computer Vision , 2005 .

[20]  Takeo Igarashi,et al.  As-rigid-as-possible shape manipulation , 2005, ACM Trans. Graph..

[21]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[22]  F. Dornaika,et al.  Mosaicking images with parallax , 2004, Signal Process. Image Commun..

[23]  David Salesin,et al.  Interactive digital photomontage , 2004, ACM Trans. Graph..

[24]  Andrea Fusiello,et al.  High resolution video mosaicing with global alignment , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[25]  Harry Shum,et al.  Correction to Construction of Panoramic Image Mosaics with Global and Local Alignment , 2001, International Journal of Computer Vision.

[26]  Richard Szeliski,et al.  Systems and Experiment Paper: Construction of Panoramic Image Mosaics with Global and Local Alignment , 2000, International Journal of Computer Vision.

[27]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[28]  Marc Alexa,et al.  Computing and Rendering Point Set Surfaces , 2003, IEEE Trans. Vis. Comput. Graph..

[29]  Richard Szeliski,et al.  Construction of panoramic mosaics with global and lo-cal alignment , 2020 .

[30]  Shmuel Peleg,et al.  Mosaicing on Adaptive Manifolds , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[31]  Gérard G. Medioni,et al.  A graph-based global registration for 2D mosaics , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[32]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[33]  Richard I. Hartley,et al.  In Defense of the Eight-Point Algorithm , 1997, IEEE Trans. Pattern Anal. Mach. Intell..

[34]  Zhengyou Zhang,et al.  Parameter estimation techniques: a tutorial with application to conic fitting , 1997, Image Vis. Comput..

[35]  Gene H. Golub,et al.  Matrix computations (3rd ed.) , 1996 .

[36]  Edward H. Adelson,et al.  A multiresolution spline with application to image mosaics , 1983, TOGS.

[37]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.