Simulation analysis of spherical panoramic mosaic

Abstract Multi-camera mosaic has become mainstream for obtaining panoramic images. Current mosaic methods based on intensity, frequency, or feature of images are limited in terms of image quality. We simulate the process of spherical panoramic mosaic and propose the corresponding calibration method based on the relationship among multiple cameras. This method is independent of image content but utilizes the orientation and position of each camera directly. Experimental results demonstrate that (1) the simulated imaging model and mosaic method are effective under different numbers of horizontal/vertical cameras; (2) that the influence of rotation angle error on panoramic mosaic can be eliminated entirely through calibration; and (3) that the influence of center offset error on panoramic mosaic is directly proportional to the center offset and inversely proportional to the object distance. This misalignment, which cannot be eliminated theoretically because the object distance is unknown, is limited when the center offset is small.

[1]  Shmuel Peleg,et al.  Seamless image stitching by minimizing false edges , 2006, IEEE Transactions on Image Processing.

[2]  Zuxun Zhang,et al.  SGM-based seamline determination for urban orthophoto mosaicking , 2016 .

[3]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[4]  Michael S. Brown,et al.  As-Projective-As-Possible Image Stitching with Moving DLT , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Kostas Daniilidis,et al.  Catadioptric Projective Geometry , 2001, International Journal of Computer Vision.

[6]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[7]  Yichen Wei,et al.  Efficient histogram-based sliding window , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Xiaoyan Zhu,et al.  A novel registration method that incorporates template matching and mutual information , 2009, 2009 IEEE International Conference on Automation and Logistics.

[9]  C. Fraser,et al.  AUTOMATED TARGET-FREE CAMERA CALIBRATION , 2014 .

[10]  H. Maas,et al.  Validation of geometric models for fisheye lenses , 2009 .

[11]  Xiaoping Liu,et al.  Automatic Registration of Multisensor Images Using an Integrated Spatial and Mutual Information (SMI) Metric , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[12]  Wolfgang Förstner,et al.  Fish-Eye-Stereo Calibration and Epipolar Rectification , 2005 .

[13]  Gyeong-il Kweon,et al.  Image-processing Based Panoramic Camera Employing Single Fisheye Lens , 2010 .

[14]  Stefanos Zafeiriou,et al.  Robust FFT-Based Scale-Invariant Image Registration with Image Gradients , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Massimo Bergamasco,et al.  Beyond virtual museums: experiencing immersive virtual reality in real museums , 2010 .

[16]  Luigi Barazzetti,et al.  Photogrammetric survey of complex geometries with low-cost software: Application to the ‘G1′ temple in Myson, Vietnam , 2011 .

[17]  H. Bakstein,et al.  Panoramic mosaicing with a 180/spl deg/ field of view lens , 2002, Proceedings of the IEEE Workshop on Omnidirectional Vision 2002. Held in conjunction with ECCV'02.

[18]  Alberto Broggi,et al.  Extensive Tests of Autonomous Driving Technologies , 2013, IEEE Transactions on Intelligent Transportation Systems.

[19]  Yonghong Jia,et al.  Registration of Panoramic/Fish-Eye Image Sequence and LiDAR Points Using Skyline Features , 2018, Sensors.

[20]  Wei Zhang,et al.  A Unified Framework for Street-View Panorama Stitching , 2016, Sensors.

[21]  Matthew A. Brown,et al.  Automatic Panoramic Image Stitching using Invariant Features , 2007, International Journal of Computer Vision.

[22]  Xinming Tang,et al.  Triple linear-array image geometry model of ZiYuan-3 surveying satellite and its validation , 2013 .

[23]  A. Ardeshir Goshtasby,et al.  Registration of multitemporal aerial optical images using line features , 2016 .

[24]  Huanfeng Shen,et al.  Multimodal registration of remotely sensed images based on Jeffrey’s divergence , 2016 .

[25]  Jeffrey S. Norris,et al.  Immersive and collaborative data visualization using virtual reality platforms , 2014, 2014 IEEE International Conference on Big Data (Big Data).

[26]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[27]  Huanfeng Shen,et al.  A robust mosaicking procedure for high spatial resolution remote sensing images , 2015 .

[28]  Valerio Pascucci,et al.  Distributed Seams for Gigapixel Panoramas , 2015, IEEE Transactions on Visualization and Computer Graphics.

[29]  Yi Zhang,et al.  Line-based Multi-Label Energy Optimization for fisheye image rectification and calibration , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Yacov Hel-Or,et al.  Piecewise-consistent color mappings of images acquired under various conditions , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[31]  Jafar Amiri Parian,et al.  Sensor modeling, self-calibration and accuracy testing of panoramic cameras and laser scanners , 2010 .

[32]  Tom Duckett,et al.  3D modeling of indoor environments by a mobile robot with a laser scanner and panoramic camera , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[33]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[34]  Jason Wither,et al.  Moving beyond the map: automated landmark based pedestrian guidance using street level panoramas , 2013, MobileHCI '13.

[35]  Chun Liu,et al.  Infrared and visible images registration using BEMD and MI , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[36]  Hans P. Moravec.  Rover Visual Obstacle Avoidance , 1981, IJCAI.

[37]  Chun-Shin Lin,et al.  Seam-line determination for image mosaicking: A technique minimizing the maximum local mismatch and the global cost , 2010 .

[38]  Michel Defrise,et al.  Symmetric Phase-Only Matched Filtering of Fourier-Mellin Transforms for Image Registration and Recognition , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[39]  Weiming Li,et al.  Single-camera panoramic stereo imaging system with a fisheye lens and a convex mirror. , 2011, Optics express.

[40]  Christian Früh,et al.  Google Street View: Capturing the World at Street Level , 2010, Computer.

[41]  Clive S. Fraser,et al.  Automated Target-Free Network Orienation and Camera Calibration , 2014 .

[42]  Fabio Remondino,et al.  Orientation and 3D modelling from markerless terrestrial images: combining accuracy with automation , 2010 .

[43]  G. Fangi,et al.  Photogrammetric Processing of Spherical Panoramas , 2013 .

[44]  Zhong Min,et al.  Panorama Stitching Based on SIFT Algorithm and Levenberg-Marquardt Optimization , 2012 .

[45]  Bernhard Rinner,et al.  A fast and mobile system for registration of low-altitude visual and thermal aerial images using multiple small-scale UAVs , 2015 .

[46]  Massimo Bertozzi,et al.  A 13,000 km Intercontinental Trip with Driverless Vehicles: The VIAC Experiment , 2013, IEEE Intelligent Transportation Systems Magazine.

[47]  Jordi Inglada,et al.  Analysis of Artifacts in Subpixel Remote Sensing Image Registration , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[48]  Richard Szeliski,et al.  Image Alignment and Stitching , 2006, Handbook of Mathematical Models in Computer Vision.