Geospatial Computer Vision Based on Multi-Modal Data - How Valuable Is Shape Information for the Extraction of Semantic Information?

In this paper, we investigate the value of different modalities and their combination for the analysis of geospatial data of low spatial resolution. For this purpose, we present a framework that allows for the enrichment of geospatial data with additional semantics based on given color information, hyperspectral information, and shape information. While the different types of information are used to define a variety of features, classification based on these features is performed using a random forest classifier. To draw conclusions about the relevance of different modalities and their combination for scene analysis, we present and discuss results which have been achieved with our framework on the MUUFL Gulfport Hyperspectral and LiDAR Airborne Data Set.

[1]  Markus Vincze,et al.  Fast semantic segmentation of 3D point clouds using a dense CRF with learned parameters , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Juliane Bendig,et al.  Low-weight and UAV-based Hyperspectral Full-frame Cameras for Monitoring Crops: Spectral Comparison with Portable Spectroradiometer Measurements , 2015 .

[3]  Uwe Soergel,et al.  HIERARCHICAL HIGHER ORDER CRF FOR THE CLASSIFICATION OF AIRBORNE LIDAR POINT CLOUDS IN URBAN AREAS , 2016 .

[4]  Markus H. Gross,et al.  Multi‐scale Feature Extraction on Point‐Sampled Surfaces , 2003, Comput. Graph. Forum.

[5]  Martin Weinmann,et al.  Comparison of belief propagation and graph-cut approaches for contextual classification of 3D lidar point cloud data , 2017, 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).

[6]  François Goulette,et al.  Paris-rue-Madame Database - A 3D Mobile Laser Scanner Dataset for Benchmarking Urban Detection, Segmentation and Classification Methods , 2014, ICPRAM.

[7]  Uwe Soergel,et al.  Contextual Classification of Full Waveform Lidar Data in the Wadden Sea , 2014, IEEE Geoscience and Remote Sensing Letters.

[8]  Michael Felsberg,et al.  Deep Projective 3D Semantic Segmentation , 2017, CAIP.

[9]  Silvio Savarese,et al.  3D Scene Understanding by Voxel-CRF , 2013, 2013 IEEE International Conference on Computer Vision.

[10]  P. Litkey,et al.  Tree species classification from fused active hyperspectral reflectance and LIDAR measurements. , 2010 .

[11]  Martin Weinmann,et al.  USING MULTI-SCALE FEATURES FOR THE 3D SEMANTIC LABELING OFAIRBORNE LASER SCANNING DATA , 2017 .

[12]  Hartmut Prautzsch,et al.  Local Versus Global Triangulations , 2001, Eurographics.

[13]  Changzhe Jiao,et al.  Multiple Instance Hyperspectral Target Characterization , 2016, ArXiv.

[14]  N. Pfeifer,et al.  Neighborhood systems for airborne laser data , 2005 .

[15]  Jon Atli Benediktsson,et al.  Recent Advances in Techniques for Hyperspectral Image Processing , 2009 .

[16]  Yi-Hsing Tseng,et al.  Airborne Dual-Wavelength LiDAR Data for Classifying Land Cover , 2014, Remote. Sens..

[17]  P. Litkey,et al.  TOWARDS AUTOMATIC SINGLE-SENSOR MAPPING BY MULTISPECTRAL AIRBORNE LASER SCANNING , 2016 .

[18]  Marc Pollefeys,et al.  Semantic3D.net: A new Large-scale Point Cloud Classification Benchmark , 2017, ArXiv.

[19]  Dimitri Lague,et al.  3D Terrestrial LiDAR data classification of complex natural scenes using a multi-scale dimensionality criterion: applications in geomorphology , 2011, ArXiv.

[20]  O. Barinova,et al.  NON-ASSOCIATIVE MARKOV NETWORKS FOR 3D POINT CLOUD CLASSIFICATION , 2010 .

[21]  C. Mallet,et al.  AIRBORNE LIDAR FEATURE SELECTION FOR URBAN CLASSIFICATION USING RANDOM FORESTS , 2009 .

[22]  Martial Hebert,et al.  Efficient 3-D scene analysis from streaming data , 2013, 2013 IEEE International Conference on Robotics and Automation.

[23]  Michael Weinmann,et al.  A Classification-Segmentation Framework for the Detection of Individual Trees in Dense MMS Point Cloud Data Acquired in Urban Areas , 2017, Remote. Sens..

[24]  Uwe Soergel,et al.  Relevance assessment of full-waveform lidar data for urban area classification , 2011 .

[25]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[26]  Martial Hebert,et al.  3-D scene analysis via sequenced predictions over points and regions , 2011, 2011 IEEE International Conference on Robotics and Automation.

[27]  Jing Huang,et al.  Point cloud labeling using 3D Convolutional Neural Network , 2016, 2016 23rd International Conference on Pattern Recognition (ICPR).

[28]  Jon Atli Benediktsson,et al.  Advances in Hyperspectral Image Classification: Earth Monitoring with Statistical Learning Methods , 2013, IEEE Signal Processing Magazine.

[29]  Martin Weinmann,et al.  Book Review–Reconstruction and Analysis of 3D Scenes: From Irregularly Distributed 3D Points to Object Classes , 2016, Photogrammetric Engineering & Remote Sensing.

[30]  John Trinder,et al.  Building detection by fusion of airborne laser scanner data and multi-spectral images : Performance evaluation and sensitivity analysis , 2007 .

[31]  C. Mallet,et al.  A structured regularization framework for spatially smoothing semantic labelings of 3D point clouds , 2017 .

[32]  Wai Yeung Yan,et al.  Urban land cover classification using airborne LiDAR data: A review , 2015 .

[33]  Markus Vincze,et al.  Enhancing Semantic Segmentation for Robotics: The Power of 3-D Entangled Forests , 2016, IEEE Robotics and Automation Letters.

[34]  Laura Chasmer,et al.  Multisensor and Multispectral LiDAR Characterization and Classification of a Forest Environment , 2016 .

[35]  S. J. Oude Elberink,et al.  IQPC 2015 TRACK: TREE SEPARATION AND CLASSIFICATION IN MOBILE MAPPING LIDAR DATA , 2015 .

[36]  A. Lucieer,et al.  Using a micro-UAV for ultra-high resolution multi-sensor observations of Antarctic moss beds , 2012 .

[37]  V. Wichmann,et al.  EVALUATING THE POTENTIAL OF MULTISPECTRAL AIRBORNE LIDAR FOR TOPOGRAPHIC MAPPING AND LAND COVER CLASSIFICATION , 2015 .

[38]  Bruno Vallet,et al.  TREES DETECTION FROM LASER POINT CLOUDS ACQUIRED IN DENSE URBAN AREAS BY A MOBILE MAPPING SYSTEM , 2012 .

[39]  Fan Zhang,et al.  Classification of airborne laser scanning data using JointBoost , 2015 .

[40]  Eyal Ben-Dor,et al.  Fusion of hyperspectral images and LiDAR data for civil engineering structure monitoring , 2010, 2010 2nd Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing.

[41]  Bernt Schiele,et al.  Comprehensive Colour Image Normalization , 1998, ECCV.

[42]  Lars Petersson,et al.  Non-associative Higher-Order Markov Networks for Point Cloud Classification , 2014, ECCV.

[43]  Joost van de Weijer,et al.  Author Manuscript, Published in "ieee Transactions on Image Processing Edge-based Color Constancy , 2022 .

[44]  J. Demantké,et al.  DIMENSIONALITY BASED SCALE SELECTION IN 3D LIDAR POINT CLOUDS , 2012 .

[45]  Przemysław Kupidura,et al.  Testing of Land Cover Classification from Multispectral Airborne Laser Scanning Data , 2016 .

[46]  Arnold W. M. Smeulders,et al.  Color-based object recognition , 1997, Pattern Recognit..

[47]  J. Niemeyer,et al.  Contextual classification of lidar data and building object detection in urban areas , 2014 .

[48]  Michael Weinmann,et al.  A Hybrid Semantic Point Cloud Classification-Segmentation Framework Based on Geometric Features and Semantic Rules , 2017, PFG – Journal of Photogrammetry, Remote Sensing and Geoinformation Science.

[49]  Martial Hebert,et al.  Scale selection for classification of point-sampled 3D surfaces , 2005, Fifth International Conference on 3-D Digital Imaging and Modeling (3DIM'05).

[50]  Konrad Schindler,et al.  FAST SEMANTIC SEGMENTATION OF 3D POINT CLOUDS WITH STRONGLY VARYING DENSITY , 2016 .

[51]  Ahmed El-Rabbany,et al.  AIRBORNE MULTISPECTRAL LIDAR DATA FOR LAND-COVER CLASSIFICATION AND LAND/WATER MAPPING USING DIFFERENT SPECTRAL INDEXES , 2016 .

[52]  Stefan Hinz,et al.  Investigation of the impact of dimensionality reduction and feature selection on the classification of hyperspectral EnMAP data , 2016, 2016 8th Workshop on Hyperspectral Image and Signal Processing: Evolution in Remote Sensing (WHISPERS).

[53]  C. Heipke,et al.  Contextual classification of point clouds using a two-stage CRF , 2015 .

[54]  Alexandre Boulch,et al.  Unstructured Point Cloud Semantic Labeling Using Deep Segmentation Networks , 2017, 3DOR@Eurographics.

[55]  Pushmeet Kohli,et al.  Spatial Inference Machines , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Juha Hyyppä,et al.  Object-based analysis of multispectral airborne laser scanner data for land cover classification and map updating , 2017 .

[57]  Antonio Criminisi,et al.  Decision Forests for Computer Vision and Medical Image Analysis , 2013, Advances in Computer Vision and Pattern Recognition.

[58]  Juha Hyyppä,et al.  MULTISPECTRAL AIRBORNE LASER SCANNING FOR AUTOMATED MAP UPDATING , 2016 .

[59]  Impyeong Lee,et al.  PERCEPTUAL ORGANIZATION OF 3D SURFACE POINTS , 2002 .

[60]  Martial Hebert,et al.  Contextual classification with functional Max-Margin Markov Networks , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[61]  Niloy J. Mitra,et al.  Estimating surface normals in noisy point cloud data , 2003, SCG '03.

[62]  James R. Lersch,et al.  Context-driven automated target detection in 3D data , 2004, SPIE Defense + Commercial Sensing.

[63]  Guihua Zhao,et al.  3D LAND COVER CLASSIFICATION BASED ON MULTISPECTRAL LIDAR POINT CLOUDS , 2016 .

[64]  Konrad Schindler,et al.  An Overview and Comparison of Smooth Labeling Methods for Land-Cover Classification , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[65]  George Vosselman,et al.  Optimizing Multiple Kernel Learning for the Classification of UAV Data , 2016, Remote. Sens..

[66]  Bruno Vallet,et al.  TerraMobilita/IQmulus Urban Point Cloud Classification Benchmark , 2014 .

[67]  Loic Landrieu,et al.  WEAKLY SUPERVISED SEGMENTATION-AIDED CLASSIFICATION OF URBAN SCENES FROM 3D LIDAR POINT CLOUDS , 2017 .