A Multi-View Dense Image Matching Method for High-Resolution Aerial Imagery Based on a Graph Network

Multi-view dense matching is a crucial process in automatic 3D reconstruction and mapping applications. In this paper, we present a robust and effective multi-view dense matching algorithm for high-resolution aerial images based on a graph network. The overlap ratio and intersection angle between image pairs are used to find candidate stereo pairs and build the graph network. A Coarse-to-Fine strategy based on an improved Semi-Global Matching algorithm is applied for disparity computation across stereo pairs. Based on the constructed graph, point clouds of base views are generated by triangulating all connected image nodes, followed by a fusion process with the average reprojection error as a priority measure. The proposed method was successfully applied in experiments on aerial image test dataset provided by the ISPRS of Vaihingen, Germany and an oblique nadir image block of Zurich, Switzerland, using three kinds of matching configurations. The proposed method was compared to other state-of-art methods, SURE and PhotoScan. The results demonstrate that the proposed method delivers matches at higher completeness, efficiency, and accuracy than the other methods tested; the RMS for average reprojection error reached the sub pixel level and the actual positioning deviation was better than 1.5 GSD.

[1]  M. Rothermel,et al.  SURE – The ifp Software for Dense Image Matching , 2013 .

[2]  Jan-Michael Frahm,et al.  Real-Time Plane-Sweeping Stereo with Multiple Sweeping Directions , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Charles T. Loop,et al.  Computing rectifying homographies for stereo vision , 1999, Proceedings. 1999 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Cat. No PR00149).

[4]  Andrea Fusiello,et al.  ACCURATE MULTIVIEW STEREO RECONSTRUCTION WITH FAST VISIBILITY INTEGRATION AND TIGHT DISPARITY BOUNDING , 2013 .

[5]  Heiko Hirschmüller,et al.  Stereo Processing by Semiglobal Matching and Mutual Information , 2008, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  A. Verri,et al.  A compact algorithm for rectification of stereo pairs , 2000 .

[7]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Radu Bogdan Rusu,et al.  3D is here: Point Cloud Library (PCL) , 2011, 2011 IEEE International Conference on Robotics and Automation.

[9]  M. Pierrot Deseilligny,et al.  APERO, AN OPEN SOURCE BUNDLE ADJUSMENT SOFTWARE FOR AUTOMATIC CALIBRATION AND ORIENTATION OF SET OF IMAGES , 2012 .

[10]  Reinhard Koch,et al.  Multi Viewpoint Stereo from Uncalibrated Video Sequences , 1998, ECCV.

[11]  Ingemar J. Cox,et al.  A maximum-flow formulation of the N-camera stereo correspondence problem , 1998, Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271).

[12]  Pascal Monasse,et al.  Three-step image rectification , 2010, BMVC.

[13]  Reinhard Koch,et al.  A simple and efficient rectification method for general motion , 1999, Proceedings of the Seventh IEEE International Conference on Computer Vision.

[14]  M. Downey,et al.  SEMI-GLOBAL MATCHING : AN ALTERNATIVE TO LIDAR FOR DSM GENERATION ? , 2010 .

[15]  K. Moffett,et al.  Remote Sens , 2015 .

[16]  N. Haala The Landscape of Dense Image Matching Algorithms , 2013 .

[17]  R. Reulke,et al.  Remote Sensing and Spatial Information Sciences , 2005 .

[18]  H. Hirschmüller Accurate and Efficient Stereo Processing by Semi-Global Matching and Mutual Information , 2005, CVPR.

[19]  Fabio Remondino,et al.  State of the art in high density image matching , 2014 .

[20]  Fabio Remondinoa,et al.  IMAGE-BASED SURFACE MEASUREMENT FOR CLOSE-RANGE HERITAGE DOCUMENTATION , 2008 .

[21]  M. Rothermel,et al.  Benchmarking High Density Image Matching for Oblique Airborne Imagery , 2014 .

[22]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[23]  Jian Sun,et al.  Guided Image Filtering , 2010, ECCV.

[24]  Heiko Hirschmüller,et al.  Semi-Global Matching-Motivation, Developments and Applications , 2011 .

[25]  Michael Goesele,et al.  Multi-View Stereo for Community Photo Collections , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[26]  Li Zhang Automatic Digital Surface Model (DSM) generation from linear array images , 2005 .

[27]  Jan-Michael Frahm,et al.  Fast Global Labeling for Real-Time Stereo Using Multiple Plane Sweeps , 2008, VMV.

[28]  Enhua Wu,et al.  Constant Time Weighted Median Filtering for Stereo Matching and Beyond , 2013, 2013 IEEE International Conference on Computer Vision.

[29]  Heiko Hirschmüller,et al.  Evaluation of Stereo Matching Costs on Images with Radiometric Differences , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[30]  Jean-Philippe Pons,et al.  High Accuracy and Visibility-Consistent Dense Multiview Stereo , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  M. Rothermel,et al.  SURE : PHOTOGRAMMETRIC SURFACE RECONSTRUCTION FROM IMAGER Y , 2013 .

[32]  M. Pierrot-Deseilligny,et al.  A MULTIRESOLUTION AND OPTIMIZATION-BASED IMAGE MATCHING APPROACH : AN APPLICATION TO SURFACE RECONSTRUCTION FROM SPOT 5-HRS STEREO IMAGERY , 2006 .

[33]  Dani Lischinski,et al.  Joint bilateral upsampling , 2007, SIGGRAPH 2007.