A graph convolutional neural network for classification of building patterns using spatial vector data

Abstract Machine learning methods, specifically, convolutional neural networks (CNNs), have emerged as an integral part of scientific research in many disciplines. However, these powerful methods often fail to perform pattern analysis and knowledge mining with spatial vector data because in most cases, such data are not underlying grid-like or array structures but can only be modeled as graph structures. The present study introduces a novel graph convolution by converting it from the vertex domain into a point-wise product in the Fourier domain using the graph Fourier transform and convolution theorem. In addition, the graph convolutional neural network (GCNN) architecture is proposed to analyze graph-structured spatial vector data. The focus of this study is the classical task of building pattern classification, which remains limited by the use of design rules and manually extracted features for specific patterns. The spatial vector data representing grouped buildings are modeled as graphs, and indices for the characteristics of individual buildings are investigated to collect the input variables. The pattern features of these graphs are directly extracted by training labeled data. Experiments confirmed that the GCNN produces satisfactory results in terms of identifying regular and irregular patterns, and thus achieves a significant improvement over existing methods. In summary, the GCNN has considerable potential for the analysis of graph-structured spatial vector data as well as scope for further improvement.

[1]  Carl Salvaggio,et al.  A Fully Convolutional Network for Semantic Labeling of 3D Point Clouds , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[2]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Maoguo Gong,et al.  Feature learning and change feature classification based on deep learning for ternary change detection in SAR images , 2017 .

[4]  Fahad Shahbaz Khan,et al.  Binary Patterns Encoded Convolutional Neural Networks for Texture Recognition and Remote Sensing Scene Classification , 2017, ArXiv.

[5]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs , 2012, IEEE Transactions on Signal Processing.

[6]  Yan Zhang,et al.  3D shape segmentation via shape fully convolutional networks , 2017, Comput. Graph..

[7]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[8]  Fan Chung,et al.  Spectral Graph Theory , 1996 .

[9]  Ben Glocker,et al.  Metric learning with spectral graph convolutions on brain connectivity networks , 2018, NeuroImage.

[10]  Mohammad Hossein Karimi,et al.  An integrated framework for linear pattern extraction in the building group generalization process , 2019 .

[11]  Inderjit S. Dhillon,et al.  Weighted Graph Cuts without Eigenvectors A Multilevel Approach , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[12]  Hendrik Herold,et al.  Analyzing building stock using topographic maps and GIS , 2009 .

[13]  Alán Aspuru-Guzik,et al.  Convolutional Networks on Graphs for Learning Molecular Fingerprints , 2015, NIPS.

[14]  L. Anselin Local Indicators of Spatial Association—LISA , 2010 .

[15]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[16]  Bo Huang,et al.  Urban land-use mapping using a deep convolutional neural network with high spatial resolution multispectral remote sensing imagery , 2018, Remote Sensing of Environment.

[17]  Timothy Baldwin,et al.  Semi-supervised User Geolocation via Graph Convolutional Networks , 2018, ACL.

[18]  Tinghua Ai,et al.  Analysis of Irregular Spatial Data with Machine Learning: Classification of Building Patterns with a Graph Convolutional Neural Network (Short Paper) , 2018, GIScience.

[19]  Gotthard Meinel,et al.  Automatic identification of building types based on topographic databases – a comparison of different data sources , 2015 .

[20]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[21]  Nathan D. Cahill,et al.  Robust Spatial Filtering With Graph Convolutional Neural Networks , 2017, IEEE Journal of Selected Topics in Signal Processing.

[22]  Shihong Du,et al.  Extracting building patterns with multilevel graph partition and building grouping , 2016 .

[23]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[24]  Pierre Vandergheynst,et al.  Wavelets on Graphs via Spectral Graph Theory , 2009, ArXiv.

[25]  Frank Canters,et al.  Mapping urban form and function at city block level using spatial metrics , 2017 .

[26]  Shihong Du,et al.  Representation and discovery of building patterns: a three-level relational approach , 2016, Int. J. Geogr. Inf. Sci..

[27]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.

[28]  Melih Basaraner,et al.  Performance of shape indices and classification schemes for characterising perceptual shape complexity of building footprints in GIS , 2017, Int. J. Geogr. Inf. Sci..

[29]  Silvio Savarese,et al.  Structural-RNN: Deep Learning on Spatio-Temporal Graphs , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Richard L. Church,et al.  UC Office of the President Recent Work Title An efficient measure of compactness for two-dimensional shapes and its application in regionalization problems Permalink , 2013 .

[31]  Shihong Du,et al.  Polygonal Clustering Analysis Using Multilevel Graph‐Partition , 2015, Trans. GIS.

[32]  Doreen S. Boyd,et al.  Automated classification metrics for energy modelling of residential buildings in the UK with open algorithms , 2020 .

[33]  Pascal Frossard,et al.  The emerging field of signal processing on graphs: Extending high-dimensional data analysis to networks and other irregular domains , 2012, IEEE Signal Processing Magazine.

[34]  Tinghua Ai,et al.  The analysis and measurement of building patterns using texton co-occurrence matrices , 2016, Int. J. Geogr. Inf. Sci..

[35]  Xiang Zhang,et al.  Template Matching and Simplification Method for Building Features Based on Shape Cognition , 2017, ISPRS Int. J. Geo Inf..

[36]  Mathias Niepert,et al.  Learning Convolutional Neural Networks for Graphs , 2016, ICML.

[37]  Pierre Vandergheynst,et al.  Geodesic Convolutional Neural Networks on Riemannian Manifolds , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[38]  Jun Chen,et al.  Automated building generalization based on urban morphology and Gestalt theory , 2004, Int. J. Geogr. Inf. Sci..

[39]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[40]  Pierre Vandergheynst,et al.  Vertex-Frequency Analysis on Graphs , 2013, ArXiv.

[41]  D. Burghardt,et al.  Proximity-based grouping of buildings in urban blocks: a comparison of four algorithms , 2015 .

[42]  Xiang Zhang,et al.  Building pattern recognition in topographic data: examples on collinear and curvilinear alignments , 2011, GeoInformatica.

[43]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[44]  Robert Weibel,et al.  An Approach for the Classification of Urban Building Structures Based on Discriminant Analysis Techniques , 2008, Trans. GIS.

[45]  Jacques Bertin,et al.  Semiology of Graphics - Diagrams, Networks, Maps , 2010 .

[46]  Guo Ren-zhong Polygon Cluster Pattern Mining Based on Gestalt Principles , 2007 .

[47]  Robert Weibel,et al.  Integrating ontological modelling and Bayesian inference for pattern classification in topographic vector data , 2009, Comput. Environ. Urban Syst..

[48]  Vijayan K. Asari,et al.  Progressively Expanded Neural Network (PEN Net) for hyperspectral image classification: A new neural network paradigm for remote sensing image analysis , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[49]  Q. Guo,et al.  On the spatial distribution of buildings for map generalization , 2018 .

[50]  Zhang Liqiang,et al.  A spatial cognition-based urban building clustering approach and its applications , 2013 .

[51]  M. Deng,et al.  Recognizing building groups for generalization: a comparative study , 2018 .

[52]  Antonio Plaza,et al.  A new deep convolutional neural network for fast hyperspectral image classification , 2017, ISPRS Journal of Photogrammetry and Remote Sensing.

[53]  Xavier Bresson,et al.  Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering , 2016, NIPS.

[54]  Volker C. Radeloff,et al.  Building patterns and landscape fragmentation in northern Wisconsin, USA , 2007, Landscape Ecology.

[55]  Esther M. Arkin,et al.  An efficiently computable metric for comparing polygonal shapes , 1991, SODA '90.

[56]  Gerald Penn,et al.  Convolutional Neural Networks for Speech Recognition , 2014, IEEE/ACM Transactions on Audio, Speech, and Language Processing.

[57]  Michael G. Rabbat,et al.  Approximating signals supported on graphs , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[58]  M. Batty The Size, Scale, and Shape of Cities , 2008, Science.

[59]  Pierre Vandergheynst,et al.  Geometric Deep Learning: Going beyond Euclidean data , 2016, IEEE Signal Process. Mag..

[60]  W. Tobler A Computer Movie Simulating Urban Growth in the Detroit Region , 1970 .

[61]  Hannes Taubenböck,et al.  Building Types’ Classification Using Shape-Based Features and Linear Discriminant Functions , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[62]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[63]  Subhas C. Misra,et al.  A multi-faceted CNN architecture for automatic classification of mobile LiDAR data and an algorithm to reproduce point cloud samples for enhanced training , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[64]  Cheng Wang,et al.  A deep learning framework for road marking extraction, classification and completion from mobile laser scanning point clouds , 2019, ISPRS Journal of Photogrammetry and Remote Sensing.

[65]  U. Lombardo Quantitative morphometric analysis of lakes using GIS: rectangularity R, ellipticity E, orientation O, and the rectangularity vs. ellipticity index, REi , 2014 .

[66]  Jason Parent,et al.  Ten compactness properties of circles: measuring shape in geography , 2010 .

[67]  Robert Weibel,et al.  A Multi-parameter Approach to Automated Building Grouping and Generalization , 2008, GeoInformatica.

[68]  José M. F. Moura,et al.  Discrete Signal Processing on Graphs: Frequency Analysis , 2013, IEEE Transactions on Signal Processing.

[69]  Jianping Wu,et al.  An Extended Minimum Spanning Tree method for characterizing local urban patterns , 2018, Int. J. Geogr. Inf. Sci..

[70]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[71]  Michael Batty,et al.  Fractal Cities: A Geometry of Form and Function , 1996 .

[72]  Jonathan Masci,et al.  Geometric Deep Learning on Graphs and Manifolds Using Mixture Model CNNs , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[73]  Xinchang Zhang,et al.  Recognition of building group patterns in topographic maps based on graph partitioning and random forest , 2018 .

[74]  Lutz Plümer,et al.  Automatic classification of building types in 3D city models , 2012, GeoInformatica.

[75]  Andrew J. Tatem,et al.  Identifying residential neighbourhood types from settlement points in a machine learning approach , 2018, Comput. Environ. Urban Syst..

[76]  Ye Zhang,et al.  A light and faster regional convolutional neural network for object detection in optical remote sensing images , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.