Assessment of the Degree of Building Damage Caused by Disaster Using Convolutional Neural Networks in Combination with Ordinal Regression

We propose a new convolutional neural networks method in combination with ordinal regression aiming at assessing the degree of building damage caused by earthquakes with aerial imagery. The ordinal regression model and a deep learning algorithm are incorporated to make full use of the information to improve the accuracy of the assessment. A new loss function was introduced in this paper to combine convolutional neural networks and ordinal regression. Assessing the level of damage to buildings can be considered as equivalent to predicting the ordered labels of buildings to be assessed. In the existing research, the problem has usually been simplified as a problem of pure classification to be further studied and discussed, which ignores the ordinal relationship between different levels of damage, resulting in a waste of information. Data accumulated throughout history are used to build network models for assessing the level of damage, and models for assessing levels of damage to buildings based on deep learning are described in detail, including model construction, implementation methods, and the selection of hyperparameters, and verification is conducted by experiments. When categorizing the damage to buildings into four types, we apply the method proposed in this paper to aerial images acquired from the 2014 Ludian earthquake and achieve an overall accuracy of 77.39%; when categorizing damage to buildings into two types, the overall accuracy of the model is 93.95%, exceeding such values in similar types of theories and methods.

[1]  Robin Spence,et al.  Validating Assessments of Seismic Damage Made from Remote Sensing , 2011 .

[2]  Nazzareno Pierdicca,et al.  Earthquake damage mapping: An overall assessment of ground surveys and VHR image change detection after L'Aquila 2009 earthquake , 2018, Remote Sensing of Environment.

[3]  Alexei A. Efros,et al.  What makes ImageNet good for transfer learning? , 2016, ArXiv.

[4]  Manfred F. Buchroithner,et al.  Identifying Collapsed Buildings Using Post-Earthquake Satellite Imagery and Convolutional Neural Networks: A Case Study of the 2010 Haiti Earthquake , 2018, Remote. Sens..

[5]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[6]  Léon Bottou,et al.  Large-Scale Machine Learning with Stochastic Gradient Descent , 2010, COMPSTAT.

[7]  Amnon Shashua,et al.  Ranking with Large Margin Principle: Two Approaches , 2002, NIPS.

[8]  Lingjia Gu,et al.  A land-cover classification method of high-resolution remote sensing imagery based on convolution neural network , 2018, Optical Engineering + Applications.

[9]  Kaiming He,et al.  Focal Loss for Dense Object Detection , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[10]  Jie Shan,et al.  A comprehensive review of earthquake-induced building damage detection with remote sensing techniques , 2013 .

[11]  Peter Reinartz,et al.  Combined Edge Segment Texture Analysis for the Detection of Damaged Buildings in Crisis Areas , 2012, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[12]  Ling Li,et al.  Ordinal Regression by Extended Binary Classification , 2006, NIPS.

[13]  Keiko Saito,et al.  Visual Damage Assessment using High-Resolution Satellite Images following the 2003 Bam, Iran, Earthquake , 2005 .

[14]  Geoffrey E. Hinton,et al.  Rectified Linear Units Improve Restricted Boltzmann Machines , 2010, ICML.

[15]  Antonio-Javier Gallego,et al.  Automatic Ship Classification from Optical Aerial Images with Convolutional Neural Networks , 2018, Remote. Sens..

[16]  Dacheng Tao,et al.  Deep Ordinal Regression Network for Monocular Depth Estimation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[17]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[18]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[19]  Ross B. Girshick,et al.  Focal Loss for Dense Object Detection , 2017, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Yifang Ban,et al.  Context-based mapping of damaged buildings from high-resolution optical satellite images , 2010 .

[21]  Michael S. Bernstein,et al.  ImageNet Large Scale Visual Recognition Challenge , 2014, International Journal of Computer Vision.

[22]  Tong Zhang,et al.  Deep Learning Based Feature Selection for Remote Sensing Scene Classification , 2015, IEEE Geoscience and Remote Sensing Letters.

[23]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[24]  G. Grünthal European macroseismic scale 1998 : EMS-98 , 1998 .

[25]  Yoshua Bengio,et al.  Understanding the difficulty of training deep feedforward neural networks , 2010, AISTATS.

[26]  George Vosselman,et al.  Multi-Resolution Feature Fusion for Image Classification of Building Damages with Convolutional Neural Networks , 2018, Remote. Sens..

[27]  Qi Wen,et al.  Quantifying Disaster Physical Damage Using Remote Sensing Data—A Technical Work Flow and Case Study of the 2014 Ludian Earthquake in China , 2017, International Journal of Disaster Risk Science.

[28]  R. Wen,et al.  Characteristics of strong motions and damage implications of MS6.5 Ludian earthquake on August 3, 2014 , 2015 .

[29]  M F Sanner,et al.  Python: a programming language for software integration and development. , 1999, Journal of molecular graphics & modelling.

[30]  David A. Landgrebe,et al.  A survey of decision tree classifier methodology , 1991, IEEE Trans. Syst. Man Cybern..

[31]  Salvatore Greco,et al.  Ordinal regression revisited: Multiple criteria ranking using a set of additive value functions , 2008, Eur. J. Oper. Res..

[32]  Bing Zhang,et al.  Spatial distribution and inducement of collapsed buildings in Yushu earthquake based on remote sensing analysis , 2010 .

[33]  Pierre Alliez,et al.  Convolutional Neural Networks for Large-Scale Remote-Sensing Image Classification , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[34]  Geoffrey E. Hinton,et al.  On the importance of initialization and momentum in deep learning , 2013, ICML.

[35]  Martín Abadi,et al.  TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems , 2016, ArXiv.

[36]  Rob Fergus,et al.  Depth Map Prediction from a Single Image using a Multi-Scale Deep Network , 2014, NIPS.

[37]  Robert J. Abrahart,et al.  Chapter 2 Data-Driven Modelling : Concepts , Approaches and Experiences , 2017 .

[38]  Gang Hua,et al.  Ordinal Regression with Multiple Output CNN for Age Estimation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[39]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[40]  Bing Zhang,et al.  Spatial distribution and inducement of collapsed buildings in Yushu earthquake based on remote sensing analysis , 2010 .

[41]  Martin Kappas,et al.  Comparison of Random Forest, k-Nearest Neighbor, and Support Vector Machine Classifiers for Land Cover Classification Using Sentinel-2 Imagery , 2017, Sensors.

[42]  Peijun Li,et al.  Urban building damage detection from very high resolution imagery using OCSVM and spatial features , 2010 .

[43]  Simon Plank,et al.  Rapid Damage Assessment by Means of Multi-Temporal SAR - A Comprehensive Review and Outlook to Sentinel-1 , 2014, Remote. Sens..

[44]  Howie Choset,et al.  xBD: A Dataset for Assessing Building Damage from Satellite Imagery , 2019, ArXiv.

[45]  Yang Shao,et al.  Detection of Urban Damage Using Remote Sensing and Machine Learning Algorithms: Revisiting the 2010 Haiti Earthquake , 2016, Remote. Sens..

[46]  Qing Wang,et al.  Object-Based Land-Cover Supervised Classification for Very-High-Resolution UAV Images Using Stacked Denoising Autoencoders , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[47]  Charles K. Huyck,et al.  Towards Rapid Citywide Damage Mapping Using Neighborhood Edge Dissimilarities in Very High-Resolution Optical Satellite Imagery—Application to the 2003 Bam, Iran, Earthquake , 2005 .

[48]  Sergey Ioffe,et al.  Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift , 2015, ICML.

[49]  Jacob Cohen A Coefficient of Agreement for Nominal Scales , 1960 .

[50]  Yun Fu,et al.  Image-Based Human Age Estimation by Manifold Learning and Locally Adjusted Robust Regression , 2008, IEEE Transactions on Image Processing.

[51]  Sebastian Ruder,et al.  An overview of gradient descent optimization algorithms , 2016, Vestnik komp'iuternykh i informatsionnykh tekhnologii.

[52]  Cheng Gang,et al.  Earthquake-collapsed building extraction from LiDAR and aerophotograph based on OBIA , 2010, The 2nd International Conference on Information Science and Engineering.

[53]  G. Brent Hall,et al.  Open Source Approaches in Spatial Data Handling , 2008 .

[54]  Frank Warmerdam,et al.  The Geospatial Data Abstraction Library , 2008 .

[55]  Curt H. Davis,et al.  Training Deep Convolutional Neural Networks for Land–Cover Classification of High-Resolution Imagery , 2017, IEEE Geoscience and Remote Sensing Letters.

[56]  B. J. Adams,et al.  IMPROVED DISASTER MANAGEMENT THROUGH POST-EARTHQUAKE BUILDING DAMAGE ASSESSMENT USING MULTITEMPORAL SATELLITE IMAGERY , 2004 .

[57]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[58]  P. Reinartz,et al.  Building damage assessment after the earthquake in Haiti using two post-event satellite stereo imagery and DSMs , 2013, Joint Urban Remote Sensing Event 2013.

[59]  ZhiQiang Chen,et al.  Structural damage detection using bi-temporal optical satellite images , 2011 .

[60]  Zhenwei Shi,et al.  Multilevel Cloud Detection in Remote Sensing Images Based on Deep Learning , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[61]  Jungho Im,et al.  Support vector machines in remote sensing: A review , 2011 .

[62]  Luisa Verdoliva,et al.  Training convolutional neural networks for semantic classification of remote sensing imagery , 2017, 2017 Joint Urban Remote Sensing Event (JURSE).