A Robust Method for Automatic Panoramic UAV Image Mosaic

This paper introduces a robust method for panoramic unmanned aerial vehicle (UAV) image mosaic. In the traditional automatic panoramic image stitching method (Autostitch), it assumes that the camera rotates about its optical centre and the group of transformations the source images may undergo is a special group of homographies. It is rare to get such ideal data in reality. In particular, remote sensing images obtained by UAV do not satisfy such an ideal situation, where the images may not be on a plane yet and even may suffer from nonrigid changes, leading to poor mosaic results. To overcome the above mentioned challenges, in this paper a nonrigid matching algorithm is introduced to the mosaic system to generate accurate feature matching on remote sensing images. We also propose a new strategy for bundle adjustment to make the mosaic system suitable for the UAV image panoramic mosaic effect. Experimental results show that our method outperforms the traditional method and some of the latest methods in terms of visual effect.

[1]  Richard Szeliski,et al.  Seamless Image Stitching of Scenes with Large Motions and Exposure Differences , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[2]  Manolis I. A. Lourakis,et al.  Is Levenberg-Marquardt the most efficient optimization algorithm for implementing bundle adjustment? , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[3]  Ming Li,et al.  Improved Seam-Line Searching Algorithm for UAV Image Mosaic with Optical Flow , 2018, Sensors.

[4]  Ruizhi Chen,et al.  A Stereo Dual-Channel Dynamic Programming Algorithm for UAV Image Stitching , 2017, Sensors.

[5]  Michael S. Brown,et al.  Constructing image panoramas using dual-homography warping , 2011, CVPR 2011.

[6]  Mei Yu,et al.  Improved WαSH Feature Matching Based on 2D-DWT for Stereo Remote Sensing Images , 2018, Sensors.

[7]  Richard Szeliski,et al.  Systems and Experiment Paper: Construction of Panoramic Image Mosaics with Global and Local Alignment , 2000, International Journal of Computer Vision.

[8]  Patrick Pérez,et al.  Poisson image editing , 2003, ACM Trans. Graph..

[9]  Di Wang,et al.  A Miniature Binocular Endoscope with Local Feature Matching and Stereo Matching for 3D Measurement and 3D Reconstruction , 2018, Sensors.

[10]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[11]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[12]  Tao Lu,et al.  Robust Face Super-Resolution via Locality-Constrained Low-Rank Representation , 2017, IEEE Access.

[13]  Jing Li,et al.  Parallax-Tolerant Image Stitching Based on Robust Elastic Warping , 2018, IEEE Transactions on Multimedia.

[14]  Sharath Pankanti,et al.  Adaptive as-natural-as-possible image stitching , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Shuicheng Yan,et al.  Common visual pattern discovery via spatially coherent correspondences , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  Michael S. Brown,et al.  As-Projective-As-Possible Image Stitching with Moving DLT , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[17]  Zhuowen Tu,et al.  Robust Point Matching via Vector Field Consensus , 2014, IEEE Transactions on Image Processing.

[18]  Jiayi Ma,et al.  Infrared and visible image fusion methods and applications: A survey , 2018, Inf. Fusion.

[19]  Martial Hebert,et al.  A spectral technique for correspondence problems using pairwise constraints , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[20]  Junjun Jiang,et al.  Robust Feature Matching for Remote Sensing Image Registration via Locally Linear Transforming , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Hazem M. El-Bakry,et al.  Image Stitching based on Feature Extraction Techniques: A Survey , 2014 .

[22]  Junjun Jiang,et al.  FusionGAN: A generative adversarial network for infrared and visible image fusion , 2019, Inf. Fusion.

[23]  Zhuowen Tu,et al.  Robust $L_{2}E$ Estimation of Transformation for Non-Rigid Registration , 2015, IEEE Transactions on Signal Processing.

[24]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[25]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[26]  Jiri Matas,et al.  Matching with PROSAC - progressive sample consensus , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[27]  Naima Kaabouch,et al.  Quantitative evaluation of image mosaicing in multiple scene categories , 2012, 2012 IEEE International Conference on Electro/Information Technology.

[28]  Vladimir Kolmogorov,et al.  Feature Correspondence Via Graph Matching: Models and Global Optimization , 2008, ECCV.

[29]  Tao Lu,et al.  Multi-Memory Convolutional Neural Network for Video Super-Resolution , 2019, IEEE Transactions on Image Processing.

[30]  Naima Kaabouch,et al.  Robust Spatial-Domain Based Super-Resolution Mosaicing of CubeSat Video Frames: Algorithm and Evaluation , 2014, Comput. Inf. Sci..

[31]  Junjun Jiang,et al.  Locality Preserving Matching , 2017, IJCAI.

[32]  Yasuyuki Matsushita,et al.  Smoothly varying affine stitching , 2011, CVPR 2011.

[33]  Gérard G. Medioni,et al.  A graph-based global registration for 2D mosaics , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[34]  Edward H. Adelson,et al.  A multiresolution spline with application to image mosaics , 1983, TOGS.

[35]  Michael Gleicher,et al.  Content-preserving warps for 3D video stabilization , 2009, ACM Trans. Graph..

[36]  Andrew W. Fitzgibbon,et al.  Bundle Adjustment - A Modern Synthesis , 1999, Workshop on Vision Algorithms.

[37]  N. Aronszajn.  Theory of Reproducing Kernels. , 1950 .

[38]  Junjun Jiang,et al.  Guided Locality Preserving Feature Matching for Remote Sensing Image Registration , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Zhengyou Zhang,et al.  Parameter estimation techniques: a tutorial with application to conic fitting , 1997, Image Vis. Comput..

[40]  Andrea Fusiello,et al.  High resolution video mosaicing with global alignment , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[41]  Zhanyi Hu,et al.  Rejecting Mismatches by Correspondence Function , 2010, International Journal of Computer Vision.

[42]  Jiri Matas,et al.  Locally Optimized RANSAC , 2003, DAGM-Symposium.

[43]  Zhuowen Tu,et al.  Regularized vector field learning with sparse approximation for mismatch removal , 2013, Pattern Recognit..

[44]  Naima Kaabouch,et al.  Hierarchical multi-level image mosaicing for autonomous navigation of UAV , 2012, Electronic Imaging.

[45]  Alan L. Yuille,et al.  Non-Rigid Point Set Registration by Preserving Global and Local Structures , 2016, IEEE Transactions on Image Processing.

[46]  Junjun Jiang,et al.  LMR: Learning a Two-Class Classifier for Mismatch Removal , 2019, IEEE Transactions on Image Processing.

[47]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  David Salesin,et al.  Interactive digital photomontage , 2004, ACM Trans. Graph..

[49]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[50]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[51]  Junchi Yan,et al.  Adaptive Discrete Hypergraph Matching , 2018, IEEE Transactions on Cybernetics.

[52]  Yoichi Sato,et al.  Shape-Preserving Half-Projective Warps for Image Stitching , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.