Geographic, geometrical and semantic reconstruction of urban scene from high resolution oblique aerial images

An effective approach is proposed for 3D urban scene reconstruction in the form of point cloud with semantic labeling. Starting from high resolution oblique aerial images, our approach proceeds through three main stages: geographic reconstruction, geometrical reconstruction and semantic reconstruction. The absolute position and orientation of all the cameras relative to the real world are recovered in the geographic reconstruction stage. Then, in the geometrical reconstruction stage, an improved multi-view stereo matching method is employed to produce 3D dense points with color and normal information by taking into account the prior knowledge of aerial imagery. Finally the point cloud is classified into three classes ( building, vegetation, and ground ) by a rule-based hierarchical approach in the semantic reconstruction step. Experiments on complex urban scene show that our proposed 3-stage approach could generate reasonable reconstruction result robustly and efficiently. By comparing our final semantic reconstruction result with the manually labeled ground truth, classification accuracies from 86.75% to 93.02% are obtained.

[1]  Pascal Fua,et al.  On benchmarking camera calibration and multi-view stereo for high resolution imagery , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Richard Szeliski,et al.  A Multi-stage Linear Approach to Structure from Motion , 2010, ECCV Workshops.

[3]  Jiann-Yeou Rau,et al.  Analysis of Oblique Aerial Images for Land Cover and Point Cloud Classification in an Urban Environment , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[4]  Vincent Lepetit,et al.  DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Florent Lafarge,et al.  Creating Large-Scale City Models from 3D-Point Clouds: A Robust Approach with Hybrid Representation , 2012, International Journal of Computer Vision.

[6]  Changchang Wu,et al.  Towards Linear-Time Incremental Structure from Motion , 2013, 2013 International Conference on 3D Vision.

[7]  Xiangguo Lin,et al.  SVM-Based Classification of Segmented Airborne LiDAR Point Clouds in Urban Areas , 2013, Remote. Sens..

[8]  Tomaso Poggio,et al.  Cooperative computation of stereo disparity , 1988 .

[9]  Richard Szeliski,et al.  Building Rome in a day , 2009, ICCV.

[10]  Marsha Jo Hannah,et al.  Computer matching of areas in stereo images. , 1974 .

[11]  Menglong Yan,et al.  Building Reconstruction From High-Resolution Multiview Aerial Imagery , 2015, IEEE Geoscience and Remote Sensing Letters.

[12]  Uwe Soergel,et al.  Conditional Random Fields for Urban Scene Classification with Full Waveform LiDAR Data , 2011, PIA.

[13]  Richard Szeliski,et al.  Towards Internet-scale multi-view stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[14]  Chengcui Zhang,et al.  A progressive morphological filter for removing nonground measurements from airborne LIDAR data , 2003, IEEE Trans. Geosci. Remote. Sens..

[15]  Andrew Zisserman,et al.  Multiple View Geometry in Computer Vision (2nd ed) , 2003 .

[16]  Franz Rottensteiner.  Advanced methods for automated object extraction from LiDAR in urban areas , 2012, 2012 IEEE International Geoscience and Remote Sensing Symposium.

[17]  Shuhan Shen,et al.  Accurate Multiple View 3D Reconstruction Using Patch-Based Stereo for Large-Scale Scenes , 2013, IEEE Transactions on Image Processing.

[18]  Daniel G. Aliaga,et al.  A Survey of Urban Reconstruction , 2013, Comput. Graph. Forum.

[19]  Richard Szeliski,et al.  A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms , 2001, International Journal of Computer Vision.

[20]  Jan-Michael Frahm,et al.  Building Rome on a Cloudless Day , 2010, ECCV.

[21]  George Vosselman,et al.  Experimental comparison of filter algorithms for bare-Earth extraction from airborne laser scanning point clouds , 2004 .

[22]  Richard Szeliski,et al.  Modeling the World from Internet Photo Collections , 2008, International Journal of Computer Vision.

[23]  Henrik Aanæs,et al.  Large Scale Multi-view Stereopsis Evaluation , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Zhanyi Hu,et al.  Efficient Large-Scale Structure From Motion by Fusing Auxiliary Imaging Information , 2015, IEEE Transactions on Image Processing.

[25]  Avideh Zakhor,et al.  Classifying urban landscape in aerial LiDAR using 3D shape analysis , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[26]  Florent Lafarge,et al.  LOD Generation for Urban Scenes , 2015, ACM Trans. Graph..

[27]  G. Sithole.  FILTERING OF LASER ALTIMETRY DATA USING A SLOPE ADAPTIVE FILTER , 2001 .

[28]  Richard Szeliski,et al.  A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[29]  Xiangguo Lin,et al.  Segmentation-Based Filtering of Airborne LiDAR Point Clouds by Progressive Densification of Terrain Segments , 2014, Remote. Sens..

[30]  Lorenzo Bruzzone,et al.  Automatic Detection and Reconstruction of Building Radar Footprints From Single VHR SAR Images , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[31]  Karl-Heinz Häfele,et al.  OGC City Geography Markup Language (CityGML) Encoding Standard , 2012 .

[32]  Lutz Plümer,et al.  CityGML – Interoperable semantic 3D city models , 2012 .

[33]  Axel Petzold,et al.  The Historical Origin of the Pulfrich Effect: A Serendipitous Astronomic Observation at the Border of the Milky Way , 2009 .

[34]  Jan-Michael Frahm,et al.  Reconstructing the world* in six days , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Florent Lafarge.  Some new research directions to explore in urban reconstruction , 2015, 2015 Joint Urban Remote Sensing Event (JURSE).

[37]  Sunil Arya,et al.  An optimal algorithm for approximate nearest neighbor searching fixed dimensions , 1998, JACM.