Semantic analysis of 3D point clouds from urban environments: ground, facades, urban objects and accessibility. (Analyse sémantique de nuages de points 3D dans le milieu urbain: sol, façades, objets urbain et accessibilité)

Most important cities in the world have very detailed 2D urban plans of streets and public spaces. These plans contain information about roads, sidewalks, facades and urban objects such as lampposts, traffic signs, bollards, trees, among others. Nowadays, several local authorities, national mapping agencies and private companies have began to consider justifiable including 3D information, navigation options and accessibility issues into urban maps. Compared to the first 3D scanning systems 30 years ago, current laser scanners are cheaper, faster and provide more accurate and denser 3D point clouds. Urban analysis from these data is difficult and tedious, and existing semi-automatic methods may not be sufficiently precise nor robust. In that sense, automatic methods for 3D urban semantic analysis are required. This thesis contributes to the field of semantic analysis of 3D point clouds from urban environments. Our methods are based on elevation images and illustrate how mathematical morphology can be exploited to develop a complete 3D processing chain including six main steps: i) filtering and preprocessing; ii) ground segmentation and accessibility analysis; iii) facade segmentation, iv) object detection; v) object segmentation; and, vi) object classification. Additionally, we have worked on the integration of our results into a large-scale production chain. In that sense, our results have been exported as 3D point clouds for visualization and modeling purposes and integrated as shapefiles into Geographical Information Systems (GIS). Our methods have been qualitative and quantitative tested in several databases from the state of the art and from TerraMobilita project. Our results show that our methods are accurate, fast and outperform other works reported in the literature on the same databases. Conclusions and perspectives for future work are discussed as well.

[1]  Beatriz Marcotegui,et al.  Point cloud segmentation towards urban ground modeling , 2009, 2009 Joint Urban Remote Sensing Event.

[2]  Kim L. Boyer,et al.  The Robust Sequential Estimator: A General Approach and its Application to Surface Organization in Range Data , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[3]  Luc Vincent,et al.  Morphological Area Openings and Closings for Grey-scale Images , 1994 .

[4]  Jesús Angulo,et al.  Morphological Bilateral Filtering and Spatially-Variant Adaptive Structuring Functions , 2011, ISMM.

[5]  Ana-Maria Cretu,et al.  Neural-network-based models of 3-D objects for virtualized reality: a comparative study , 2006, IEEE Transactions on Instrumentation and Measurement.

[6]  Christian Menkens,et al.  EasyWheel - A Mobile Social Navigation and Support System for Wheelchair Users , 2011, 2011 Eighth International Conference on Information Technology: New Generations.

[7]  Fernand Meyer,et al.  From connected operators to levelings , 1998 .

[8]  Jitendra Malik,et al.  Scale-Space and Edge Detection Using Anisotropic Diffusion , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[9]  Luc Vincent,et al.  Morphological grayscale reconstruction in image analysis: applications and efficient algorithms , 1993, IEEE Trans. Image Process..

[10]  Christian Lantuéjoul,et al.  Geodesic methods in quantitative image analysis , 1984, Pattern Recognit..

[11]  Bahman Soheilian,et al.  Semi-Automatic Road/Pavement Modeling using Mobile Laser Scanning , 2013 .

[12]  Etienne Decencière,et al.  Region Growing Structuring Elements and New Operators Based on Their Shape , 2011 .

[13]  Jean Serra,et al.  Image Analysis and Mathematical Morphology , 1983 .

[14]  Petros Maragos,et al.  Threshold Superposition in Morphological Image Analysis Systems , 1990, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Andrés Serna,et al.  Urban accessibility diagnosis from mobile laser scanning data , 2013 .

[16]  Jorge Eduardo Hernández Londoño Analyse morphologique d'images pour la modélisation d'environnements urbains , 2009 .

[17]  Jesús Angulo,et al.  Spatially-Variant Area Openings for Reference-Driven Adaptive Contour Preserving Filtering , 2014, 2014 22nd International Conference on Pattern Recognition.

[18]  Martyna Poreba,et al.  Assessing the accuracy of land-based mobile laser scanning data , 2012 .

[19]  Martyna Poreba,et al.  RANSAC algorithm and elements of graph theory for automatic plane detection in 3D point clouds , 2012 .

[20]  Bernd Fritzke,et al.  A Growing Neural Gas Network Learns Topologies , 1994, NIPS.

[21]  Thomas Martinetz,et al.  'Neural-gas' network for vector quantization and its application to time-series prediction , 1993, IEEE Trans. Neural Networks.

[22]  Tomaso A. Poggio,et al.  Machine Learning, Machine Vision, and the Brain , 1999, AI Mag..

[23]  Michael Cramer,et al.  The DGPF-Test on Digital Airborne Camera Evaluation - Over- view and Test Design , 2010 .

[24]  Uwe Soergel,et al.  ANALYSIS OF FULL-WAVEFORM LIDAR DATA FOR CLASSIFICATION OF URBAN AREAS , 2008 .

[25]  Bruno Vallet,et al.  TerraMobilita/iQmulus urban point cloud analysis benchmark , 2015, Comput. Graph..

[26]  Jungho Im,et al.  Support vector machines in remote sensing: A review , 2011 .

[27]  Tania Landes,et al.  AUTOMATIC SEGMENTATION OF BUILDING FACADES USING TERRESTRIAL LASER DATA , 2007 .

[28]  Robert Hecht,et al.  Data fusion of extremely high resolution aerial imagery and LiDAR data for automated railroad centre line reconstruction , 2011 .

[29]  Reinhard Klein,et al.  Shape Recognition in 3D Point-Clouds , 2008 .

[30]  Wolfgang Förstner,et al.  Curb reconstruction using Conditional Random Fields , 2010, 2010 IEEE Intelligent Vehicles Symposium.

[31]  Andrés Serna,et al.  Segmentation of elongated objects using attribute profiles and area stability: Application to melanocyte segmentation in engineered skin , 2014, Pattern Recognit. Lett..

[32]  Florent Lafarge,et al.  Creating Large-Scale City Models from 3D-Point Clouds: A Robust Approach with Hybrid Representation , 2012, International Journal of Computer Vision.

[33]  Bahman Soheilian,et al.  Road side detection and reconstruction using LIDAR sensor , 2013, 2013 IEEE Intelligent Vehicles Symposium (IV).

[34]  N. Otsu A threshold selection method from gray level histograms , 1979 .

[35]  George Vosselman,et al.  Tree modelling from mobile laser scanning data‐sets , 2011 .

[36]  François Goulette,et al.  Extracting Cylinders in Full 3D Data Using a Random Sampling Method and the Gaussian Image , 2001, VMV.

[37]  Georges Voronoi Nouvelles applications des paramètres continus à la théorie des formes quadratiques. Premier mémoire. Sur quelques propriétés des formes quadratiques positives parfaites. , 1908 .

[38]  Ronald Jones,et al.  Attribute Openings, Thinnings, and Granulometries , 1996, Comput. Vis. Image Underst..

[39]  Rafael C. González,et al.  Digital image processing, 3rd Edition , 2008 .

[40]  Jean-Charles Pinoli,et al.  General Adaptive Neighborhood Mathematical Morphology , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[41]  D. Hubel,et al.  Receptive fields, binocular interaction and functional architecture in the cat's visual cortex , 1962, The Journal of physiology.

[42]  Martial Hebert,et al.  Onboard contextual classification of 3-D point clouds with learned high-order Markov Random Fields , 2009, 2009 IEEE International Conference on Robotics and Automation.

[43]  Masahiko Nagai,et al.  Discovering repetitive patterns in facade images using a RANSAC-style algorithm , 2014 .

[44]  Li Gang,et al.  Procedural Modeling of Urban Road Network , 2010, 2010 International Forum on Information Technology and Applications.

[45]  J. Demantké,et al.  DIMENSIONALITY BASED SCALE SELECTION IN 3D LIDAR POINT CLOUDS , 2012 .

[46]  Robert B. Fisher,et al.  Modelling Objects having Quadric Surfaces Incorporating Geometric cCnstraints , 1998, ECCV.

[47]  Alireza Bab-Hadiashar,et al.  Range image segmentation using surface selection criterion , 2006, IEEE Transactions on Image Processing.

[48]  Swarup Medasani,et al.  Automatic recognition of diverse 3-D objects and analysis of large urban scenes using ground and aerial LIDAR sensors , 2010, CLEO/QELS: 2010 Laser Science to Photonic Applications.

[49]  Timo Honkela,et al.  On the Quantization Error in SOM vs. VQ: A Critical and Systematic Study , 2009, WSOM.

[50]  C. Lantuéjoul,et al.  On the use of the geodesic metric in image analysis , 1981 .

[51]  Etienne Decencière,et al.  Efficient Geodesic Attribute Thinnings Based on the Barycentric Diameter , 2012, Journal of Mathematical Imaging and Vision.

[52]  Jiri Matas,et al.  Robust wide-baseline stereo from maximally stable extremal regions , 2004, Image Vis. Comput..

[53]  Vladimir G. Kim,et al.  Shape-based recognition of 3D point clouds in urban environments , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[54]  Jon Atli Benediktsson,et al.  Advanced directional mathematical morphology for the detection of the road network in very high resolution remote sensing images , 2010, Pattern Recognit. Lett..

[55]  Jacques G. Verly,et al.  Adaptive mathematical morphology for range imagery , 1993, IEEE Trans. Image Process..

[56]  Andrés Serna,et al.  Adaptive parameter tuning for morphological segmentation of building facade images , 2012, 2012 Proceedings of the 20th European Signal Processing Conference (EUSIPCO).

[57]  Jean Paul Frédéric Serra Viscous Lattices , 2005, Journal of Mathematical Imaging and Vision.

[58]  J. Niemeyer,et al.  Contextual classification of lidar data and building object detection in urban areas , 2014 .

[59]  Ahmad Kamal Aijazi,et al.  Segmentation Based Classification of 3D Urban Point Clouds: A Super-Voxel Based Approach with Evaluation , 2013, Remote. Sens..

[60]  Omer Rashid,et al.  Users Helping Users: User Generated Content to Assist Wheelchair Users in an Urban Environment , 2010, 2010 Ninth International Conference on Mobile Business and 2010 Ninth Global Mobility Roundtable (ICMB-GMR).

[61]  K. Tansey,et al.  Backscatter coefficient as an attribute for the classification of full-waveform airborne laser scanning data in urban areas , 2010 .

[62]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[63]  Petros Maragos,et al.  Overview of adaptive morphology: Trends and perspectives , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[64]  Jesús Angulo,et al.  Riemannian mathematical morphology , 2014, Pattern Recognit. Lett..

[65]  Mutlu Avci,et al.  A Hierarchical Classificaton of Landsat Tm Imagery for Landcover Mapping , 2004 .

[66]  Beatriz Marcotegui,et al.  Morphological segmentation of building façade images , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[67]  Jing Xiao,et al.  Fusion of airborne laserscanning point clouds and images for supervised and unsupervised scene classification , 2014 .

[68]  Thomas J. Ayres,et al.  Sidewalk Potential Trip Points: A Method for Characterizing Walkways , 2006 .

[69]  Uwe Soergel,et al.  Relevance assessment of full-waveform lidar data for urban area classification , 2011 .

[70]  Boris Jutzi,et al.  Semantic 3D scene interpretation: A framework combining optimal neighborhood size selection with relevant features , 2014 .

[71]  Boris Jutzi,et al.  Feature relevance assessment for the semantic interpretation of 3D point cloud data , 2013 .

[72]  Hugues Talbot,et al.  Efficient complete and incomplete path openings and closings , 2007, Image Vis. Comput..

[73]  Alexander M. Bronstein,et al.  Stable volumetric features in deformable shapes , 2012, Comput. Graph..

[74]  F. Goulette,et al.  An integrated on-board laser range sensing system for on-the-way city and road modelling , 2006 .

[75]  Jean-Emmanuel Deschaud Traitements de nuages de points denses et modélisation 3D d'environnements par système mobile LiDAR/Caméra. (Dense point cloud processing and 3D environment modeling from LiDAR/Camera mobile system) , 2010 .

[76]  Andrés Serna,et al.  Attribute Controlled Reconstruction and Adaptive Mathematical Morphology , 2013, ISMM.

[77]  David H. Douglas,et al.  ALGORITHMS FOR THE REDUCTION OF THE NUMBER OF POINTS REQUIRED TO REPRESENT A DIGITIZED LINE OR ITS CARICATURE , 1973 .

[78]  Philippe Salembier,et al.  Flat zones filtering, connected operators, and filters by reconstruction , 1995, IEEE Trans. Image Process..

[79]  Julius Ziegler,et al.  Team AnnieWAY's autonomous system for the 2007 DARPA Urban Challenge , 2008, J. Field Robotics.

[80]  Alexander M. Bronstein,et al.  Are MSER Features Really Interesting? , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[81]  Jean Paul Frédéric Serra,et al.  Connectivity on Complete Lattices , 1998, Journal of Mathematical Imaging and Vision.

[82]  Karsten Schulz,et al.  CONTEXT-BASED URBAN TERRAIN RECONSTRUCTION FROM IMAGES AND VIDEOS , 2012 .

[83]  B. Marcotegui,et al.  FILTERING OF ARTIFACTS AND PAVEMENT SEGMENTATION FROM MOBILE LIDAR DATA , 2009 .

[84]  Etienne Decencière,et al.  Geodesic Attributes Thinnings and Thickenings , 2011, ISMM.

[85]  George Vosselman,et al.  Airborne and terrestrial laser scanning , 2011, Int. J. Digit. Earth.

[86]  Bertrand Douillard,et al.  On the segmentation of 3D LIDAR point clouds , 2011, 2011 IEEE International Conference on Robotics and Automation.

[87]  David G. Lowe,et al.  Shape Descriptors for Maximally Stable Extremal Regions , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[88]  P. Soille,et al.  Automated basin delineation from digital elevation models using mathematical morphology , 1990 .

[89]  R M Rangayyan,et al.  Feature enhancement of film mammograms using fixed and adaptive neighborhoods. , 1984, Applied optics.

[90]  Xin He,et al.  Parallel Algorithms for Gray-Scale Digitized Picture Component Labeling on a Mesh-Connected Computer , 1994, J. Parallel Distributed Comput..

[91]  Ronald Jones,et al.  Connected Filtering and Segmentation Using Component Trees , 1999, Comput. Vis. Image Underst..

[92]  Nikos Paragios,et al.  Segmentation of building facades using procedural shape priors , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[93]  D. Grigillo,et al.  Urban Object Extraction from Digital Surface Model and Digital Aerial Images , 2012 .

[94]  Philippe Salembier,et al.  Connected operators and pyramids , 1993, Optics & Photonics.

[95]  N. David,et al.  Stereopolis II: A multi-purpose and multi-sensor 3D mobile mapping system for street visualisation and 3D metrology , 2014 .

[96]  L. E. Link,et al.  Airborne laser topographic mapping results , 1984 .

[97]  George Vosselman,et al.  Recognizing basic structures from mobile laser scanning data for road inventory studies , 2011 .

[98]  David Nistér,et al.  Linear Time Maximally Stable Extremal Regions , 2008, ECCV.

[99]  Bruno Vallet,et al.  TerraMobilita/IQmulus Urban Point Cloud Classification Benchmark , 2014 .

[100]  F. Meyer IMAGE SEGMENTATION BASED ON VISCOUS FLOODING SIMULATION , 2002 .

[101]  Jos B. T. M. Roerdink Adaptivity and group invariance in mathematical morphology , 2009, 2009 16th IEEE International Conference on Image Processing (ICIP).

[102]  Herbert Edelsbrunner,et al.  Three-dimensional alpha shapes , 1992, VVS.

[103]  E. Denis,et al.  TOWARDS ROAD MODELLING FROM TERRESTRIAL LASER POINTS , 2010 .

[104]  Per-Erik Forssén,et al.  Maximally Stable Colour Regions for Recognition and Matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[105]  Ben Taskar,et al.  Discriminative learning of Markov random fields for segmentation of 3D scan data , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[106]  Andrés Serna,et al.  Detection, segmentation and classification of 3D urban objects using mathematical morphology and supervised learning , 2014 .

[107]  Manuel Mazo,et al.  Proposal for an Ambient Assisted Wheelchair (A2W) , 2010, 2010 IEEE International Symposium on Industrial Electronics.

[108]  G. Matheron Random Sets and Integral Geometry , 1976 .

[109]  François Goulette,et al.  Paris-rue-Madame Database - A 3D Mobile Laser Scanner Dataset for Benchmarking Urban Detection, Segmentation and Classification Methods , 2014, ICPRAM.

[110]  Atsuyuki Okabe,et al.  Spatial Tessellations: Concepts and Applications of Voronoi Diagrams , 1992, Wiley Series in Probability and Mathematical Statistics.

[111]  Ralph R. Martin,et al.  Robust Segmentation of Primitives from Range Data in the Presence of Geometric Degeneracy , 2001, IEEE Trans. Pattern Anal. Mach. Intell..

[112]  Pierre Soille,et al.  Morphological Image Analysis: Principles and Applications , 2003 .

[113]  F. Meyer,et al.  ISMM 05 Special Issue : Image ltering using morphological amoebas , 2006 .

[114]  J. Roerdink,et al.  MATHEMATICAL MORPHOLOGY AND ITS APPLICATIONS TO IMAGE PROCESSING , 1994 .

[115]  Andrew W. Fitzgibbon,et al.  An Experimental Comparison of Range Image Segmentation Algorithms , 1996, IEEE Trans. Pattern Anal. Mach. Intell..

[116]  S. Kolski,et al.  Detection, prediction, and avoidance of dynamic obstacles in urban environments , 2008, 2008 IEEE Intelligent Vehicles Symposium.

[117]  George Vosselman,et al.  Mapping curbstones in airborne and mobile laser scanning data , 2012, Int. J. Appl. Earth Obs. Geoinformation.

[118]  Frank Y. Shih,et al.  Adaptive mathematical morphology for edge linking , 2004, Inf. Sci..

[119]  Serge Beucher,et al.  Numerical residues , 2007, Image Vis. Comput..

[120]  Jon Atli Benediktsson,et al.  A new approach for the morphological segmentation of high-resolution satellite imagery , 2001, IEEE Trans. Geosci. Remote. Sens..

[121]  Hongbin Zha,et al.  Segmentation and classification of range image from an intelligent vehicle in urban environment , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[122]  Pierre Soille,et al.  Constrained connectivity for hierarchical image partitioning and simplification , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[123]  Pierre Soille,et al.  Adaptive Morphological Filtering Using Similarities Based on Geodesic Time , 2008, DGCI.

[124]  Michel Bilodeau,et al.  SMIL: Simple Morphological Image Library , 2013 .

[125]  Frédo Durand,et al.  Bilateral Filtering: Theory and Applications: Series: Foundations and Trends® in Computer Graphics and Vision , 2009 .

[126]  Makoto Nagao,et al.  Region extraction and shape analysis in aerial photographs , 1979 .

[127]  Nikolaus Studnicka,et al.  DOCUMENTATION OF COMPLEX FACADES AND CITY MODELLING THROUGH THE COMBINATION OF LASERSCANNING AND PHOTOGRAMMETRY , 2004 .

[128]  Ayman Habib,et al.  An adaptive approach for the segmentation and extraction of planar and linear/cylindrical features from laser scanning data , 2014 .

[129]  Ben Gorte PLANAR FEATURE EXTRACTION IN TERRESTRIAL LASER SCANS USING GRADIENT BASED RANGE IMAGE SEGMENTATION , 2007 .

[130]  Jim R. Parker,et al.  Algorithms for image processing and computer vision , 1996 .

[131]  George Vosselman,et al.  DETECTION OF CURBSTONES IN AIRBORNE LASER SCANNING DATA , 2009 .