Machine learning applications for building structural design and performance assessment: State-of-the-art review

Abstract Machine learning models have been shown to be useful for predicting and assessing structural performance, identifying structural condition and informing preemptive and recovery decisions by extracting patterns from data collected via various sources and media. This paper presents a review of the historical development and recent advances in the application of machine learning to the area of building structural design and performance assessment. To this end, an overview of machine learning theory and the most relevant algorithms is provided with the goal of identifying problems suitable for machine learning and the appropriate models to use. The machine learning applications in building structural design and performance assessment are then reviewed in four main categories: (1) predicting structural response and performance, (2) interpreting experimental data and formulating models to predict component-level structural properties, (3) information retrieval using images and written text and (4) recognizing patterns in structural health monitoring data. The challenges of bringing machine learning into structural engineering practice are identified, and future research opportunities are discussed.

[1]  Emil Pitkin,et al.  Peeking Inside the Black Box: Visualizing Statistical Learning With Plots of Individual Conditional Expectation , 2013, 1309.6392.

[2]  Nhat-Duc Hoang,et al.  Punching shear capacity estimation of FRP-reinforced concrete slabs using a hybrid machine learning approach , 2016 .

[3]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[4]  Khalid M. Mosalam,et al.  Deep Transfer Learning for Image‐Based Structural Damage Recognition , 2018, Comput. Aided Civ. Infrastructure Eng..

[5]  Zdeněk P. Bažant,et al.  Statistical linear regression analysis of prediction models for creep and shrinkage , 1983 .

[6]  Reginald DesRoches,et al.  Machine Vision-Enhanced Postearthquake Inspection , 2011, J. Comput. Civ. Eng..

[7]  Lorenzo Rosasco,et al.  Are Loss Functions All the Same? , 2004, Neural Computation.

[8]  Nhat-Duc Hoang,et al.  Automatic Detection of Concrete Spalling Using Piecewise Linear Stochastic Gradient Descent Logistic Regression and Image Texture Analysis , 2019, Complex..

[9]  Stephanie German Paal,et al.  Machine Learning-Based Backbone Curve Model of Reinforced Concrete Columns Subjected to Cyclic Loading Reversals , 2018, J. Comput. Civ. Eng..

[10]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[11]  Markus König,et al.  Achievements and Challenges in Machine Vision-Based Inspection of Large Concrete Structures , 2014 .

[12]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[13]  Javad Vaseghi Amiri,et al.  Prediction of lateral confinement coefficient in reinforced concrete columns using M5′ machine learning method , 2013, KSCE Journal of Civil Engineering.

[14]  Prabhat Hajela,et al.  Neurobiological computational models in structural analysis and design , 1991 .

[15]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[16]  Kevin P. Murphy,et al.  Machine learning - a probabilistic perspective , 2012, Adaptive computation and machine learning series.

[17]  James H. Garrett,et al.  Knowledge-Based Modeling of Material Behavior with Neural Networks , 1992 .

[18]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[19]  Seong-Hoon Hwang,et al.  Data-driven machine-learning-based seismic failure mode identification of reinforced concrete shear walls , 2020 .

[20]  Curt B. Haselton,et al.  Calibration of Model to Simulate Response of Reinforced Concrete Beam-Columns to Collapse , 2016 .

[21]  Nagiza F. Samatova,et al.  Theory-Guided Data Science: A New Paradigm for Scientific Discovery from Data , 2016, IEEE Transactions on Knowledge and Data Engineering.

[22]  Dimitrios G. Lignos,et al.  Assessment of structural damage detection methods for steel structures using full-scale experimental data and nonlinear analysis , 2018, Bulletin of Earthquake Engineering.

[23]  Han Sun,et al.  Estimating aftershock collapse vulnerability using mainshock intensity, structural response and physical damage indicators , 2017 .

[24]  Ehsan Khojastehfar,et al.  Collapse fragility curve development using Monte Carlo simulation and artificial neural network , 2014 .

[25]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[26]  Helmut Krawinkler,et al.  Deterioration Modeling of Steel Components in Support of Collapse Prediction of Steel Moment Frames under Earthquake Loading , 2011 .

[27]  Huan Luo,et al.  A locally weighted machine learning model for generalized prediction of drift capacity in seismic vulnerability assessments , 2019, Comput. Aided Civ. Infrastructure Eng..

[28]  Fatih Kucuksubasi,et al.  Transfer Learning-Based Crack Detection by Autonomous UAVs , 2018, Proceedings of the 35th International Symposium on Automation and Robotics in Construction (ISARC).

[29]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[30]  Trevor Hastie,et al.  Multi-class AdaBoost ∗ , 2009 .

[31]  Yu Zhang,et al.  Pattern recognition approach to assess the residual structural capacity of damaged tall buildings , 2019, Structural Safety.

[32]  Hoon Sohn,et al.  Damage diagnosis using time series analysis of vibration signals , 2001 .

[33]  Henry V. Burton,et al.  Deep learning-based classification of earthquake-impacted buildings using textual damage descriptions , 2019, International Journal of Disaster Risk Reduction.

[34]  Hojjat Adeli,et al.  A neural dynamics model for structural optimization—Theory , 1995 .

[35]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[36]  Charles R. Farrar,et al.  Machine learning algorithms for damage detection under operational and environmental variability , 2011 .

[37]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[38]  Zdenek P. Bazant,et al.  Bayesian Statistical Prediction of Concrete Creep and Shrinkage , 1984 .

[39]  S. Masri,et al.  Identification of Nonlinear Dynamic Systems Using Neural Networks , 1993 .

[40]  Jong-Su Jeon,et al.  Classification of failure mode and prediction of shear strength for reinforced concrete beam-column joints using machine learning techniques , 2018 .

[41]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[42]  Alberto Carpinteri,et al.  A truncated statistical model for analyzing the size-effect on tensile strength of concrete structures , 1995 .

[43]  Yoram Reich,et al.  Machine Learning Techniques for Civil Engineering Problems , 1997 .

[44]  R. Tibshirani.  Regression Shrinkage and Selection via the Lasso , 1996 .

[45]  Mia Hubert,et al.  Robust statistics for outlier detection , 2011, WIREs Data Mining Knowl. Discov..

[46]  Alejandro Betancourt,et al.  Automatic detection of building typology using deep learning methods on street level images , 2020 .

[47]  J. Freidman,et al.  Multivariate adaptive regression splines , 1991 .

[48]  Henry V. Burton,et al.  Response surface analysis and optimization of controlled rocking steel braced frames , 2018, Bulletin of Earthquake Engineering.

[49]  Tian Han,et al.  A Tale of Three Probabilistic Families: Discriminative, Descriptive and Generative Models , 2018, Quarterly of Applied Mathematics.

[50]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[51]  Henry V. Burton,et al.  Parameterized fragility functions for controlled rocking steel braced frames , 2018, Engineering Structures.

[52]  Jian Li,et al.  Vision‐Based Fatigue Crack Detection of Steel Structures Using Video Feature Tracking , 2018, Comput. Aided Civ. Infrastructure Eng..

[53]  Reginald DesRoches,et al.  Automated Damage Index Estimation of Reinforced Concrete Columns for Post-Earthquake Evaluations , 2015 .

[54]  Matthew R. Hanlon,et al.  DesignSafe: New Cyberinfrastructure for Natural Hazards Engineering , 2017 .

[55]  Barry J. Goodno,et al.  Metamodel-based regional vulnerability estimate of irregular steel moment-frame structures subjected to earthquake events , 2012 .

[56]  Nasser M. Nasrabadi,et al.  Pattern Recognition and Machine Learning , 2006, Technometrics.

[57]  Joachim M. Buhmann,et al.  Unsupervised Learning without Overfitting: Empirical Risk Approximation as an Induction Principle for Reliable Clustering , 1999 .

[58]  Nhat-Duc Hoang,et al.  Predicting ultimate bond strength of corroded reinforcement and surrounding concrete using a metaheuristic optimized least squares support vector regression model , 2019, Neural Computing and Applications.

[59]  Dimitrios G. Lignos,et al.  Use of Wavelet-Based Damage-Sensitive Features for Structural Damage Diagnosis Using Strong Motion Data , 2011 .

[60]  Jerome P. Lynch,et al.  Decentralization of wireless monitoring and control technologies for smart civil structures , 2002 .

[61]  Paul A. Viola,et al.  Rapid object detection using a boosted cascade of simple features , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[62]  R. D. Vanluchene,et al.  Neural Networks in Structural Engineering , 1990 .

[63]  J J Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities. , 1982, Proceedings of the National Academy of Sciences of the United States of America.

[64]  John Messner,et al.  StructNet: A Neural Network for Structural System Selection , 1994 .

[65]  Hojjat Adeli,et al.  A novel machine learning‐based algorithm to detect damage in high‐rise building structures , 2017 .

[66]  Reginald DesRoches,et al.  Rapid entropy-based detection and properties measurement of concrete spalling with machine vision for post-earthquake safety assessments , 2012, Adv. Eng. Informatics.

[67]  Ioannis Brilakis,et al.  Visual retrieval of concrete crack properties for automated post-earthquake structural safety evaluation , 2011 .

[68]  Hans-Peter Kriegel,et al.  A Density-Based Algorithm for Discovering Clusters in Large Spatial Databases with Noise , 1996, KDD.

[69]  D. W. Hobbs,et al.  The compressive strength of concrete: a statistical approach to failure , 1972 .

[70]  Arthur E. Hoerl,et al.  Ridge Regression: Biased Estimation for Nonorthogonal Problems , 2000, Technometrics.

[71]  Manolis Papadrakakis,et al.  Developing fragility curves based on neural network IDA predictions , 2011 .

[72]  Robert Tibshirani,et al.  The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd Edition , 2001, Springer Series in Statistics.

[73]  Young-Jin Cha,et al.  Vision-based detection of loosened bolts using the Hough transform and support vector machines , 2016 .

[74]  Nikhil Muralidhar,et al.  Incorporating Prior Domain Knowledge into Deep Neural Networks , 2018, 2018 IEEE International Conference on Big Data (Big Data).

[75]  Jürgen Schmidhuber,et al.  LSTM can Solve Hard Long Time Lag Problems , 1996, NIPS.

[76]  Henry V. Burton,et al.  Classifying earthquake damage to buildings using machine learning , 2020 .

[77]  Paul W. Fieguth,et al.  A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure , 2015, Adv. Eng. Informatics.

[78]  P. Torkzadeh,et al.  A machine-learning approach for structural damage detection using least square support vector machine based on a new combinational kernel function , 2016 .

[79]  Anuj Karpatne,et al.  Physics-guided Neural Networks (PGNN): An Application in Lake Temperature Modeling , 2017, ArXiv.

[80]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[81]  Raphael T. Haftka,et al.  Surrogate Model-Based Optimization Framework: A Case Study in Aerospace Design , 2007, Evolutionary Computation in Dynamic and Uncertain Environments.

[82]  Zdeněk P. Bažant,et al.  Improved prediction model for time-dependent deformations of concrete: Part 2—Basic creep , 1991 .

[83]  Zhiming Zhang,et al.  Structural damage identification via physics-guided machine learning: a methodology integrating pattern recognition with finite element model updating , 2020, Structural Health Monitoring.

[84]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[85]  Rainer Lienhart,et al.  An extended set of Haar-like features for rapid object detection , 2002, Proceedings. International Conference on Image Processing.

[86]  Fooad Karimi Ghaleh Jough,et al.  Prediction of seismic collapse risk of steel moment frame mid-rise structures by meta-heuristic algorithms , 2016, Earthquake Engineering and Engineering Vibration.

[87]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[88]  George C. Lee,et al.  A Structural Damage Neural Network Monitoring System , 1994 .

[89]  C. John Yoon,et al.  Neural Network Approaches to Aid Simple Truss Design Problems , 1994 .

[90]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[91]  Bolei Zhou,et al.  Learning Deep Features for Discriminative Localization , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[92]  Hojjat Adeli,et al.  Perceptron Learning in Engineering Design , 2008 .

[93]  Shahram Pezeshk,et al.  On the application of machine learning techniques to derive seismic fragility curves , 2019, Computers & Structures.

[94]  Yoshua Bengio,et al.  Generative Adversarial Nets , 2014, NIPS.

[95]  Vladimir Vapnik,et al.  Support-vector networks , 2004, Machine Learning.

[96]  John S. Gero,et al.  Effect of Representation on the Performance of Neural Networks in Structural Engineering Applications , 1994 .

[97]  James H. Garrett,et al.  Use of neural networks in detection of structural damage , 1992 .

[98]  Reginald DesRoches,et al.  Statistical models for shear strength of RC beam‐column joints using machine‐learning techniques , 2014 .

[99]  Konstantinos Morfidis,et al.  Seismic parameters' combinations for the optimum prediction of the damage state of R/C buildings using neural networks , 2017, Adv. Eng. Softw..

[100]  Joong-Koo Kim,et al.  Improved prediction model for time-dependent deformations of concrete: Part 1-Shrinkage , 1991 .

[101]  Takeshi Nagata,et al.  Building-damage detection method based on machine learning utilizing aerial photographs of the Kumamoto earthquake , 2020 .

[102]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[103]  Honglan Huang,et al.  Classification of in-plane failure modes for reinforced concrete frames with infills using machine learning , 2019, Journal of Building Engineering.

[104]  Ioannis Brilakis,et al.  Progressive 3D reconstruction of infrastructure with videogrammetry , 2011 .

[105]  H. Burton,et al.  A machine learning framework for assessing post-earthquake structural safety , 2018 .

[106]  H. Akaike.  A new look at the statistical model identification , 1974 .

[107]  John W. Wallace,et al.  Reconstructing seismic response demands across multiple tall buildings using kernel‐based machine learning methods , 2019, Structural Control and Health Monitoring.

[108]  Bernd Droge,et al.  Bootstrap and Cross-Validation Estimates of the Prediction Error for Linear Regression Models , 1984 .