Geometric-constrained multi-view image matching method based on semi-global optimization

Abstract Targeting at a reliable image matching of multiple remote sensing images for the generation of digital surface models, this paper presents a geometric-constrained multi-view image matching method, based on an energy minimization framework. By employing a geometrical constraint, the cost value of the energy function was calculated from multiple images, and the cost value was aggregated in an image space using a semi-global optimization approach. A homography transform parameter calculation method is proposed for fast calculation of projection pixel on each image when calculating cost values. It is based on the known interior orientation parameters, exterior orientation parameters, and a given elevation value. For an efficient and reliable processing of multiple remote sensing images, the proposed matching method was performed via a coarse-to-fine strategy through image pyramid. Three sets of airborne remote sensing images were used to evaluate the performance of the proposed method. Results reveal that the multi-view image matching can improve matching reliability. Moreover, the experimental results show that the proposed method performs better than traditional methods.

[1]  Roberto Cipolla,et al.  Multiview Stereo via Volumetric Graph-Cuts and Occlusion Robust Photo-Consistency , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Pradeep K. Srivastava,et al.  Robust Stereo Image Matching for Spaceborne Imagery , 2007, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Bingcai Zhang,et al.  NEXT GENERATION AUTOMATIC TERRAIN EXTRACTION USING MICROSOFT ULTRACAM IMAGERY , 2007 .

[5]  U. Helava Digital correlation in photogrammetric instruments , 1978 .

[6]  A. Gruen ADAPTIVE LEAST SQUARES CORRELATION: A POWERFUL IMAGE MATCHING TECHNIQUE , 1985 .

[7]  J. Gong,et al.  Triangulation of Well-Defined Points as a Constraint for Reliable Image Matching , 2005 .

[8]  In-So Kweon,et al.  Adaptive Support-Weight Approach for Correspondence Search , 2006, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  A. Gruen Development and Status of Image Matching in Photogrammetry , 2012 .

[10]  D. Scharstein,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, Proceedings IEEE Workshop on Stereo and Multi-Baseline Vision (SMBV 2001).

[11]  F. Bignone Processing of stereo scanner : from stereo plotter to pixel factory , 2003 .

[12]  Bo Wu,et al.  A Triangulation-based Hierarchical Image Matching Method for Wide-Baseline Images , 2011 .

[13]  Daniel P. Huttenlocher,et al.  Efficient Belief Propagation for Early Vision , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[14]  Martina L. Hobi,et al.  Accuracy Assessment of Digital Surface Models Based on WorldView-2 and ADS80 Stereo Remote Sensing Data , 2012, Sensors.

[15]  Li Zhang,et al.  Multi-image matching for DSM generation from IKONOS imagery , 2006 .

[16]  Lukas Zebedin,et al.  Towards 3D map generation from digital aerial images , 2006 .

[17]  T. Pock,et al.  Point Clouds: Lidar versus 3D Vision , 2010 .

[18]  Bo Wu,et al.  Integrated point and edge matching on poor textural images constrained by self-adaptive triangulations , 2012 .

[19]  G. P. Otto,et al.  "Region-growing" algorithm for matching of terrain images , 1989, Image Vis. Comput..

[20]  Heiko Hirschmüller,et al.  Improvements in real-time correlation-based stereo vision , 2001, CVPR 2001.

[21]  Long Quan,et al.  Match Propagation for Image-Based Modeling and Rendering , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[22]  Mark R. Stevens,et al.  Methods for Volumetric Reconstruction of Visual Scenes , 2004, International Journal of Computer Vision.

[23]  Ramin Zabih,et al.  Non-parametric Local Transforms for Computing Visual Correspondence , 1994, ECCV.

[24]  de,et al.  ASPECTS OF THE DSM PRODUCTION WITH HIGH RESOLUTION IMAGES , 2008 .

[25]  Qing Zhu,et al.  Multiple close‐range image matching based on a self‐adaptive triangle constraint , 2010 .

[26]  C. Heipke,et al.  Multi-view dense matching supported by triangular meshes , 2011 .

[27]  Arko Lucieer,et al.  Assessing the Accuracy of Georeferenced Point Clouds Produced via Multi-View Stereopsis from Unmanned Aerial Vehicle (UAV) Imagery , 2012, Remote. Sens..

[28]  Takeo Kanade,et al.  A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[29]  ARNO KNAPITSCH,et al.  Tanks and temples , 2017, ACM Trans. Graph..

[30]  Demetri Terzopoulos,et al.  Regularization of Inverse Visual Problems Involving Discontinuities , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Olga Veksler,et al.  Fast approximate energy minimization via graph cuts , 2001, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[32]  Heung-Kyu Lee,et al.  Extraction of digital elevation models from satellite stereo images through stereo matching based on epipolarity and scene geometry , 2003, Image Vis. Comput..

[33]  Zuxun Zhang,et al.  Automatic Registration of Multi-Source Imagery Based on Global Image Matching , 2000 .

[34]  Heiko Hirschmüller,et al.  Evaluation of Stereo Matching Costs on Images with Radiometric Differences , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  K. Jacobsen,et al.  Comparison of Matching Algorithms for DSM Generation in Urban Areas from Ikonos Imagery , 2010 .

[36]  Aaron F. Bobick,et al.  Large Occlusion Stereo , 1999, International Journal of Computer Vision.

[37]  Li Yan,et al.  A Multi-View Dense Image Matching Method for High-Resolution Aerial Imagery Based on a Graph Network , 2016, Remote. Sens..

[38]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[39]  Qing Zhu,et al.  Propagation strategies for stereo image matching based on the dynamic triangle constraint , 2007 .

[40]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .