Analysis of Oblique Aerial Images for Land Cover and Point Cloud Classification in an Urban Environment

In addition to aerial imagery, point clouds are important remote sensing data in urban environment studies. It is essential to extract semantic information from both images and point clouds for such purposes; thus, this study aims to automatically classify 3-D point clouds generated using oblique aerial imagery (OAI)/vertical aerial imagery (VAI) into various urban object classes, such as roof, facade, road, tree, and grass. A multicamera airborne imaging system that can simultaneously acquire VAI and OAI is suggested. The acquired small-format images contain only three RGB spectral bands and are used to generate photogrammetric point clouds through a multiview-stereo dense matching technique. To assign each 3-D point cloud to a corresponding urban object class, we first analyzed the original OAI through object-based image analyses. A rule-based hierarchical semantic classification scheme that utilizes spectral information and geometry- and topology-related features was developed, in which the object height and gradient features were derived from the photogrammetric point clouds to assist in the detection of elevated objects, particularly for the roof and facade. Finally, the photogrammetric point clouds were classified into the aforementioned five classes. The classification accuracy was assessed on the image space, and four experimental results showed that the overall accuracy is between 82.47% and 91.8%. In addition, visual and consistency analyses were performed to demonstrate the proposed classification scheme's feasibility, transferability, and reliability, particularly for distinguishing elevated objects from OAI, which has a severe occlusion effect, image-scale variation, and ambiguous spectral characteristics.

[1]  Robert M. Haralick,et al.  Textural Features for Image Classification , 1973, IEEE Trans. Syst. Man Cybern..

[2]  F. Rottensteiner,et al.  Classification of urban LiDAR data using conditional random field and random forests , 2013, Joint Urban Remote Sensing Event 2013.

[3]  Markus Gerke,et al.  Supervised and unsupervised MRF based 3D scene classification in multiple view airborne oblique images , 2013 .

[4]  Ruiliang Pu,et al.  Object-based urban detailed land cover classification with high spatial resolution IKONOS imagery , 2011 .

[5]  Görres Grenzdörffer,et al.  Photogrammetric image acquisition and image analysis of oblique imagery , 2008 .

[6]  Luc Van Gool,et al.  Dense matching of multiple wide-baseline views , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[7]  Christian Früh,et al.  Automated texture mapping of 3D city models with oblique aerial imagery , 2004, Proceedings. 2nd International Symposium on 3D Data Processing, Visualization and Transmission, 2004. 3DPVT 2004..

[8]  Jaewook Jung,et al.  Results of the ISPRS benchmark on urban object detection and 3D building reconstruction , 2014 .

[9]  W. Cohen,et al.  Lidar Remote Sensing of the Canopy Structure and Biophysical Properties of Douglas-Fir Western Hemlock Forests , 1999 .

[10]  M. Gerke,et al.  Automatic Structural Seismic Damage Assessment with Airborne Oblique Pictometry© Imagery , 2011 .

[11]  Arno Schäpe,et al.  Multiresolution Segmentation : an optimization approach for high quality multi-scale image segmentation , 2000 .

[12]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[13]  Xiaojun Yang,et al.  Urban remote sensing : monitoring, synthesis and modeling in the urban environment , 2011 .

[14]  M. Rothermel,et al.  SURE : PHOTOGRAMMETRIC SURFACE RECONSTRUCTION FROM IMAGER Y , 2013 .

[15]  Richard Szeliski,et al.  Towards Internet-scale multi-view stereo , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[16]  George Vosselman,et al.  EVIDENCE OF WALLS IN OBLIQUE IMAGES FOR AUTOMATIC VERIFICATION OF BUILDINGS , 2010 .

[17]  Jiann-Yeou Rau,et al.  True orthophoto generation of built-up areas using multi-view images , 2002 .

[18]  Roberto Cipolla,et al.  Modelling and Interpretation of Architecture from Several Images , 2004, International Journal of Computer Vision.

[19]  M. Gerke Dense matching in high resolution oblique airborne images , 2009 .

[20]  Jean Ponce,et al.  Accurate, Dense, and Robust Multiview Stereopsis , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Manchun Li,et al.  A methodology for true orthorectification of large-scale urban aerial images and automatic detection of building occlusions using digital surface model , 2003, IGARSS 2003. 2003 IEEE International Geoscience and Remote Sensing Symposium. Proceedings (IEEE Cat. No.03CH37477).

[22]  T. H. Kolbe,et al.  ON THE USE OF GEOMETRIC AND SEMANTIC MODELS FOR COMPONENT-BASED BUILDING RECONSTRUCTION , 1999 .

[23]  Changjae Kim,et al.  New Methodologies for True Orthophoto Generation , 2007 .

[24]  M. Herold,et al.  Object-oriented mapping and analysis of urban land use / cover using IKONOS data , 2002 .

[25]  Ronen Basri,et al.  Direct visibility of point sets , 2007, ACM Trans. Graph..

[26]  Imdad Ali Rizvi,et al.  Object-Based Image Analysis of High-Resolution Satellite Images Using Modified Cloud Basis Function Neural Network and Probabilistic Relaxation Labeling Process , 2011, IEEE Transactions on Geoscience and Remote Sensing.

[27]  John Trinder,et al.  Building detection by fusion of airborne laser scanner data and multi-spectral images : Performance evaluation and sensitivity analysis , 2007 .

[28]  Ahmed Mohamed Hamruni The use of oblique and vertical images for 3D urban modelling , 2010 .

[29]  J. Brasington,et al.  Object-based land cover classification using airborne LiDAR , 2008 .

[30]  Jing Xiao,et al.  Fusion of airborne laserscanning point clouds and images for supervised and unsupervised scene classification , 2014 .

[31]  F. Ackermann Airborne laser scanning : present status and future expectations , 1999 .

[32]  M. Ramsey,et al.  Monitoring urban land cover change: An expert system approach to land cover classification of semiarid to arid urban centers , 2001 .

[33]  S. Franklin,et al.  Detecting translational landslide scars using segmentation of Landsat ETM+ and DEM data in the northern Cascade Mountains, British Columbia , 2003 .

[34]  T. Esch,et al.  Urban structure type characterization using hyperspectral remote sensing and height information , 2012 .

[35]  Guifeng Zhang,et al.  A Scale-Synthesis Method for High Spatial Resolution Remote Sensing Image Segmentation , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[36]  A. Baumgartner,et al.  Fusion of LIDAR data and aerial imagery for automatic reconstruction of building surfaces , 2003, 2003 2nd GRSS/ISPRS Joint Workshop on Remote Sensing and Data Fusion over Urban Areas.

[37]  Avideh Zakhor,et al.  Urban Landscape Classification System Using Airborne LiDAR , 2008 .

[38]  Albert Rango,et al.  Texture and Scale in Object-Based Analysis of Subdecimeter Resolution Unmanned Aerial Vehicle (UAV) Imagery , 2009, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Norbert Pfeifer,et al.  A Comparison of Evaluation Techniques for Building Extraction From Airborne Laser Scanning , 2009, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[40]  Heiko Hirschmüller,et al.  Stereo Vision in Structured Environments by Consistent Semi-Global Matching , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[41]  N. Haala The Landscape of Dense Image Matching Algorithms , 2013 .

[42]  Norbert Pfeifer,et al.  A Comprehensive Automated 3D Approach for Building Extraction, Reconstruction, and Regularization from Airborne Laser Scanning Point Clouds , 2008, Sensors.

[43]  Ming Zhong,et al.  Object-Based Classification of Urban Areas Using VHR Imagery and Height Points Ancillary Data , 2012, Remote. Sens..

[44]  Jean-Philippe Pons,et al.  Towards high-resolution large-scale multi-view stereo , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  Florent Lafarge,et al.  Creating Large-Scale City Models from 3D-Point Clouds: A Robust Approach with Hybrid Representation , 2012, International Journal of Computer Vision.

[46]  Peter Axelsson,et al.  Processing of laser scanner data-algorithms and applications , 1999 .

[47]  Patricia Gober,et al.  Per-pixel vs. object-based classification of urban land cover extraction using high spatial resolution imagery , 2011, Remote Sensing of Environment.

[48]  Norbert Haala,et al.  An update on automatic 3D building reconstruction , 2010 .

[49]  Albert K. Chong,et al.  Object-Based Classification of Ikonos Imagery for Mapping Large-Scale Vegetation Communities in Urban Areas , 2007, Sensors.

[50]  Gong Wei,et al.  Multi-wavelength canopy LiDAR for remote sensing of vegetation: Design and system performance , 2012 .

[51]  Alan H. Strahler,et al.  Deriving and validating Leaf Area Index (LAI) at multiple spatial scales through lidar remote sensing: a case study in Sierra National Forest, CA , 2014 .

[52]  A. Bachelor GLOSSARY OF TERMS GLOSSARY OF TERMS , 2010 .

[53]  Qihao Weng,et al.  A survey of image classification methods and techniques for improving classification performance , 2007 .

[54]  Eyal Ofek,et al.  Validation of vector data using oblique images , 2008, GIS '08.

[55]  B. Devereux,et al.  An efficient image segmentation algorithm for landscape analysis , 2004 .

[56]  Markus Gerke,et al.  FITTING OF PARAMETRIC BUILDING MODELS TO OBLIQUE AERIAL IMAGES , 2012 .

[57]  George Vosselman,et al.  Line matching in oblique airborne images to support automatic verification of building outlines , 2010 .

[58]  C. Brenner,et al.  Virtual city models from laser altimeter and 2D map data , 1999 .

[59]  Richard Szeliski,et al.  Manhattan-world stereo , 2009, CVPR.

[60]  George Vosselman,et al.  Report: ISPRS Comparison of Filters , 2003 .

[61]  Belur V. Dasarathy,et al.  Urban remote sensing using multiple data sets: Past, present, and future , 2005, Inf. Fusion.

[62]  Austin Troy,et al.  Object-based land cover classification of shaded areas in high spatial resolution imagery of urban areas: A comparison study , 2009 .

[63]  Thomas Blaschke,et al.  Geographic Object-Based Image Analysis – Towards a new paradigm , 2014, ISPRS journal of photogrammetry and remote sensing : official publication of the International Society for Photogrammetry and Remote Sensing.

[64]  C. Burnett,et al.  A multi-scale segmentation/object relationship modelling methodology for landscape analysis , 2003 .

[65]  Primo Zingaretti,et al.  Complete classification of raw LIDAR data and 3D reconstruction of buildings , 2006, Pattern Analysis and Applications.

[66]  K. Wenzel,et al.  A comparison of dense matching algorithms for scaled surface reconstruction using stereo camera rigs , 2013 .

[67]  Markus Gerke,et al.  The ISPRS benchmark on urban object classification and 3D building reconstruction , 2012 .

[68]  Thomas Blaschke,et al.  Object based image analysis for remote sensing , 2010 .

[69]  Joanne C. White,et al.  Lidar sampling for large-area forest characterization: A review , 2012 .

[70]  M. Molenaar,et al.  Objects with fuzzy spatial extent , 1999 .

[71]  T. Blaschke,et al.  Automated classification of landform elements using object-based image analysis , 2006 .

[72]  Avideh Zakhor,et al.  Tree Detection in Urban Regions Using Aerial Lidar and Image Data , 2007, IEEE Geoscience and Remote Sensing Letters.

[73]  Sarah Smith-Voysey,et al.  Integrating building footprints and LiDAR elevation data to classify roof structures and visualise buildings , 2009, Computers, Environment and Urban Systems.

[74]  Leif Kobbelt,et al.  Character animation from 2D pictures and 3D motion data , 2007, TOGS.

[75]  Y. Tseng,et al.  Incremental segmentation of lidar point clouds with an octree‐structured voxel space , 2011 .

[76]  George Vosselman,et al.  Building extraction from oblique airborne imagery based on robust façade detection , 2012 .