Automated detection of buildings from single VHR multispectral images using shadow information and graph cuts

Abstract In this study, we propose a novel methodology for automated detection of buildings from single very-high-resolution (VHR) multispectral images. The methodology uses the principal evidence of buildings: the shadows that they cast. We model the directional spatial relationship between buildings and their shadows using a recently proposed probabilistic landscape approach. An effective shadow post-processing step is developed to focus on landscapes that belong to building regions. The building regions are detected using an original two-level graph theory approach. In the first level, each shadow region is addressed separately, and building regions are identified via iterative graph cuts designed in two-label partitioning. The final building regions are characterised in a second level in which the previously labelled building regions are subjected to a single-step multi-label graph optimisation performed over the entire image domain. Numerical assessments performed on 16 VHR GeoEye-1 images demonstrate that the proposed approach is highly robust and reliable. A distinctive specialty of the proposed approach is its applicability to buildings with diverse characteristics as well as to VHR images with significantly different illumination properties.

[1]  Nikos Paragios,et al.  Recognition-Driven Two-Dimensional Competing Priors Toward Automatic and Accurate Building Detection , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[2]  J. Peng,et al.  Model and context‐driven building extraction in dense urban aerial images , 2005 .

[3]  J. Shan,et al.  CLASS-GUIDED BUILDING EXTRACTION FROM IKONOS IMAGERY , 2003 .

[4]  Pooya Sarabandi,et al.  Shadow detection and radiometric restoration in satellite high resolution images , 2004, IGARSS 2004. 2004 IEEE International Geoscience and Remote Sensing Symposium.

[5]  Jon Atli Benediktsson,et al.  Classification and feature extraction for remote sensing images from urban areas based on morphological transformations , 2003, IEEE Trans. Geosci. Remote. Sens..

[6]  Jefferey A. Shufelt,et al.  Performance Evaluation and Analysis of Monocular Building Extraction From Aerial Imagery , 1999, IEEE Trans. Pattern Anal. Mach. Intell..

[7]  Parvaneh Saeedi,et al.  Three-Dimensional Polygonal Building Model Estimation From Single Satellite Images , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[8]  Claus Brenner,et al.  Interpretation of Urban Surface Models Using 2D Building Information , 1998, Comput. Vis. Image Underst..

[9]  Uwe Soergel,et al.  Building Detection From One Orthophoto and High-Resolution InSAR Data Using Conditional Random Fields , 2011, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[10]  Rudi Goossens,et al.  3D building reconstruction based on given ground plan information and surface models extracted from spaceborne imagery , 2012 .

[11]  Ramakant Nevatia,et al.  Building Detection and Description from a Single Intensity Image , 1998, Comput. Vis. Image Underst..

[12]  M. Molenaar,et al.  Objects with fuzzy spatial extent , 1999 .

[13]  Jack Bresenham,et al.  Algorithm for computer control of a digital plotter , 1965, IBM Syst. J..

[14]  George Vosselman,et al.  Reconstruction of 3D building models from aerial images and maps , 2003 .

[15]  R. Bruce Irvin,et al.  Methods for exploiting the relationship between buildings and their shadows in aerial imagery , 1989, IEEE Trans. Syst. Man Cybern..

[16]  X. Briottet,et al.  Shadow detection in very high spatial resolution aerial images: A comparative study , 2013 .

[17]  Allen R. Hanson,et al.  Recognition and reconstruction of buildings from multiple aerial images , 2003, Comput. Vis. Image Underst..

[18]  Ramakant Nevatia,et al.  Detecting buildings in aerial images , 1988, Comput. Vis. Graph. Image Process..

[19]  Norbert Haala,et al.  An update on automatic 3D building reconstruction , 2010 .

[20]  Konrad Schindler,et al.  An Overview and Comparison of Smooth Labeling Methods for Land-Cover Classification , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Helmut Mayer,et al.  Automatic Object Extraction from Aerial Imagery - A Survey Focusing on Buildings , 1999, Comput. Vis. Image Underst..

[22]  Mustafa Turker,et al.  A model-based approach for automatic building database updating from high-resolution space imagery , 2012 .

[23]  J. Chris McGlone,et al.  Fusion of HYDICE hyperspectral data with panchromatic imagery for cartographic feature extraction , 1999, IEEE Trans. Geosci. Remote. Sens..

[24]  Vladimir Kolmogorov,et al.  "GrabCut": interactive foreground extraction using iterated graph cuts , 2004, ACM Trans. Graph..

[25]  P. Zingaretti,et al.  Performance evaluation of automated approaches to building detection in multi-source aerial data , 2010 .

[26]  Vladimir Kolmogorov,et al.  An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision , 2004, IEEE Trans. Pattern Anal. Mach. Intell..

[27]  Mustafa Turker,et al.  Detection of collapsed buildings caused by the 1999 Izmit, Turkey earthquake through digital analysis of post-event aerial photographs , 2004 .

[28]  You Hongjian,et al.  3D building reconstruction from aerial CCD image and sparse laser sample data , 2006 .

[29]  Jan-Peter Muller,et al.  A technique for 3D building reconstruction , 1998 .

[30]  Xin Yang,et al.  Man‐made object detection in aerial images using multi‐stage level set evolution , 2007 .

[31]  D. Alspach.  A gaussian sum approach to the multi-target identification-tracking problem , 1975, Autom..

[32]  Rama Chellappa,et al.  Delineating buildings by grouping lines with MRFs , 1996, IEEE Trans. Image Process..

[33]  Çaglar Senaras,et al.  Automated Detection of Arbitrarily Shaped Buildings in Complex Environments From Monocular VHR Optical Satellite Imagery , 2013, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Kuo-Liang Chung,et al.  Efficient Shadow Detection of Color Aerial Images Based on Successive Thresholding Scheme , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[35]  Ramakant Nevatia,et al.  Using Perceptual Organization to Extract 3-D Structures , 1989, IEEE Trans. Pattern Anal. Mach. Intell..

[36]  Ramakant Nevatia,et al.  Automatic description of complex buildings from multiple images , 2004, Comput. Vis. Image Underst..

[37]  Henri Maître,et al.  3-D Reconstruction of Urban Scenes from Aerial Stereo Imagery: A Focusing Strategy , 1999, Comput. Vis. Image Underst..

[38]  John Trinder,et al.  Building detection by fusion of airborne laser scanner data and multi-spectral images : Performance evaluation and sensitivity analysis , 2007 .

[39]  Geoffrey J. McLachlan,et al.  Finite Mixture Models , 2019, Annual Review of Statistics and Its Application.

[40]  Jean-Yves Tourneret,et al.  High-Resolution Optical and SAR Image Fusion for Building Database Updating , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Theodosios Pavlidis,et al.  Use of Shadows for Extracting Buildings in Aerial Images , 1990, Comput. Vis. Graph. Image Process..

[42]  Jordi Inglada,et al.  Automatic recognition of man-made objects in high resolution optical remote sensing images by SVM classification of geometric image features , 2007 .

[43]  Victor J. D. Tsai,et al.  A comparative study on shadow compensation of color aerial images in invariant color models , 2006, IEEE Transactions on Geoscience and Remote Sensing.

[44]  Olga Veksler,et al.  Fast Approximate Energy Minimization via Graph Cuts , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[45]  Heinz Rüther,et al.  Application of snakes and dynamic programming optimisation technique in modeling of buildings in informal settlement areas , 2002 .

[46]  Armin B. Cremers,et al.  Extracting Buildings from Aerial Images Using Hierarchical Aggregation in 2D and 3D , 1998, Comput. Vis. Image Underst..

[47]  Mustafa Turker,et al.  An adaptive fuzzy-genetic algorithm approach for building detection using high-resolution satellite images , 2013, Comput. Environ. Urban Syst..

[48]  Jon Atli Benediktsson,et al.  A new approach for the morphological segmentation of high-resolution satellite imagery , 2001, IEEE Trans. Geosci. Remote. Sens..

[49]  Kim L. Boyer,et al.  A system to detect houses and residential street networks in multispectral satellite images , 2005, Comput. Vis. Image Underst..

[50]  Florence Tupin,et al.  Detection of building outlines based on the fusion of SAR and optical features , 2003 .

[51]  Hichem Sahli,et al.  A Stochastic Framework for the Identification of Building Rooftops Using a Single Remote Sensing Image , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Dong-Chen He,et al.  Rule-Based Classification of a Very High Resolution Image in an Urban Environment Using Multispectral Segmentation Guided by Cartographic Data , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[53]  Jefferey A. Shufelt,et al.  Fusion of monocular cues to detect man-made structures in aerial imagery , 1993 .

[54]  Franklin César Flores,et al.  Automatic shadow segmentation in aerial color images , 2003, 16th Brazilian Symposium on Computer Graphics and Image Processing (SIBGRAPI 2003).

[55]  Terry Caelli,et al.  Building Detection Using Bayesian Networks , 2000, Int. J. Pattern Recognit. Artif. Intell..

[56]  David Declercq,et al.  Three-dimensional building detection and modeling using a statistical approach , 2001, IEEE Trans. Image Process..

[57]  Ramakant Nevatia,et al.  Detection and Modeling of Buildings from Multiple Aerial Images , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[58]  Luis Ángel Ruiz Fernández,et al.  Evaluation of Automatic Building Detection Approaches Combining High Resolution Images and LiDAR Data , 2011, Remote. Sens..

[59]  J. Chris McGlone,et al.  Projective and object space geometry for monocular building extraction , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[60]  M. Fradkin,et al.  Building Detection from Multiple Aerial Images in Dense Urban Areas , 2001, Comput. Vis. Image Underst..

[61]  Hamid Abrishami Moghaddam,et al.  Automatic urban building boundary extraction from high resolution aerial images using an innovative model of active contours , 2010, Int. J. Appl. Earth Obs. Geoinformation.

[62]  R. Bruce Irvin,et al.  Methods For Exploiting The Relationship Between Buildings And Their Shadows In Aerial Imagery , 1989, Photonics West - Lasers and Applications in Science and Engineering.

[63]  I. Dowman,et al.  Data fusion of high-resolution satellite imagery and LiDAR data for automatic building extraction * , 2007 .

[64]  Cem Ünsalan,et al.  Urban-Area and Building Detection Using SIFT Keypoints and Graph Theory , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[65]  Kadim Tasdemir,et al.  Automatic Detection and Segmentation of Orchards Using Very High Resolution Imagery , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[66]  Christophe Vestri.  Using range data in automatic modeling of buildings , 2006, Image Vis. Comput..

[67]  Matthieu Cord,et al.  Building Detection and Reconstruction from Mid- and High-Resolution Aerial Imagery , 1998, Comput. Vis. Image Underst..

[68]  Bruno Vallet,et al.  Building footprint database improvement for 3D reconstruction: A split and merge approach and its evaluation , 2011 .

[69]  C. Fraser,et al.  Automatic Detection of Residential Buildings Using LIDAR Data and Multispectral Imagery , 2010 .

[70]  Emmanuel P. Baltsavias,et al.  Object extraction and revision by image analysis using existing geodata and knowledge: current status and steps towards operational systems☆ , 2004 .

[71]  C. Brenner.  Building reconstruction from images and laser scanning , 2005 .

[72]  Yerach Doytsher,et al.  MONOCULAR RIGHT-ANGLE BUILDING HYPOTHESIS GENERATION IN REGULARIZED URBAN AREAS BY POSE CLUSTERING , 2003 .

[73]  O. Tournaire,et al.  An efficient stochastic approach for building footprint extraction from digital elevation models , 2010 .

[74]  Wolfgang Förstner,et al.  Towards automatic building extraction from high-resolution digital elevation models , 1995 .

[75]  Jing Peng,et al.  An improved snake model for building detection from urban aerial images , 2005, Pattern Recognit. Lett..

[76]  Matthieu Cord,et al.  Accurate Building Structure Recovery from High Resolution Aerial Imagery , 2001, Comput. Vis. Image Underst..

[77]  George Vosselman,et al.  Building extraction from oblique airborne imagery based on robust façade detection , 2012 .

[78]  Huseyin Gokhan Akcay,et al.  Building detection using directional spatial constraints , 2010, 2010 IEEE International Geoscience and Remote Sensing Symposium.

[79]  Curt H. Davis,et al.  A combined fuzzy pixel-based and object-based approach for classification of high-resolution multispectral data over urban areas , 2003, IEEE Trans. Geosci. Remote. Sens..

[80]  Arnold W. M. Smeulders,et al.  Color-based object recognition , 1997, Pattern Recognit..

[81]  Bodo Rosenhahn,et al.  Segment-based building detection with conditional random fields , 2011, 2011 Joint Urban Remote Sensing Event.

[82]  Allen R. Hanson,et al.  The Ascender System: Automated Site Modeling from Multiple Aerial Images , 1998, Comput. Vis. Image Underst..

[83]  Taejung Kim,et al.  Development of a graph-based approach for building detection , 1999, Image Vis. Comput..

[84]  Yun Zhang,et al.  Optimisation of building detection in satellite images by combining multispectral classification and texture filtering , 1999 .

[85]  Josiane Zerubia,et al.  Automatic Building Extraction from DEMs using an Object Approach and Application to the 3D-city Modeling , 2008 .

[86]  N. Otsu.  A threshold selection method from gray level histograms , 1979 .

[87]  Olivier Jamet,et al.  Extraction and textural characterization of above-ground areas from aerial stereo pairs: a quality assessment , 1998 .

[88]  Fatos T. Yarman-Vural,et al.  Building Detection With Decision Fusion , 2013, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.