An engineer's guide to eXplainable Artificial Intelligence and Interpretable Machine Learning: Navigating causality, forced goodness, and the false perception of inference

Abstract While artificial intelligence (AI), and by extension machine learning (ML), continues to be adopted in parallel engineering disciplines, the integration of AI/ML into the structural engineering domain remains minutus. This resistance towards AI and ML primarily stems from two folds: 1) the fact that coding/programming is not a frequent element in structural engineering curricula, and 2) these methods are displayed as blackboxes; the opposite of that often favored by structural engineering education and industry (i.e., testing, empirical analysis, numerical simulation, etc.). Naturally, structural engineers are reluctant to leverage AI/ML during their tenure as such technology is viewed as opaque. In the rare instances of engineers adopting AI/ML, a clear emphasis is displayed towards chasing goodness metrics to imply “viable” inference. However, and just like the notion of correlation does not infer causation, forced goodness is prone to indicate a false sense of inference. To overcome this challenge, this paper advocates for a modern form of AI, one that is humanly explainable; thereby eXplainable Artificial Intelligence (XAI) and interpretable machine learning (IML). Thus, this work dives into the inner workings of a typical analysis to demystify how AI/ML model predictions can be evaluated and interpreted through a collection of agnostic methods (e.g., feature importance, partial dependence plots, feature interactions, SHAP (SHapley Additive exPlanations), and surrogates) via a thorough examination of a case study carried out on a comprehensive database compiled on reinforced concrete (RC) beams strengthened with fiber-reinforced polymer (FRP) composite laminates. In this case study, three algorithms, namely: Extreme Gradient Boosted Trees (ExGBT), Light gradient boosted trees (LGBT), and Keras Deep Neural Networks (KDNN), are applied to predict the maximum moment capacity of FRP-strengthened beams and the propensity of the FRP system to fail under various mechanisms. Finally, a philosophical engineering perspective into future research directions pertaining to this domain is presented and articulated.

[1]  Arnaud Castel,et al.  RC beams strengthened with NSM CFRP rods and modeling of peeling-off failure , 2010 .

[2]  J. Pearl.  Causal inference in statistics: An overview , 2009 .

[3]  M. Z. Naser,et al.  Evaluating structural response of concrete-filled steel tubular columns through machine learning , 2020, Journal of Building Engineering.

[4]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[5]  M. Z. Naser,et al.  Observational Analysis of Fire-Induced Spalling of Concrete through Ensemble Machine Learning and Surrogate Modeling , 2021, Journal of Materials in Civil Engineering.

[6]  Honglan Huang,et al.  Classification of in-plane failure modes for reinforced concrete frames with infills using machine learning , 2019, Journal of Building Engineering.

[7]  Hod Lipson,et al.  Age-fitness pareto optimization , 2010, GECCO '10.

[8]  Brad Boehmke,et al.  Interpretable Machine Learning , 2019 .

[9]  Adrian Barbu,et al.  Prediction of the crack condition of highway pavements using machine learning models , 2019 .

[10]  Roberto Turrin,et al.  Performance of recommender algorithms on top-n recommendation tasks , 2010, RecSys '10.

[11]  Cao Vu Dung,et al.  Autonomous concrete crack detection using deep fully convolutional neural network , 2019, Automation in Construction.

[12]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[13]  Thanasis Triantafillou,et al.  Strengthening of RC beams with epoxy-bonded fibre-composite materials , 1992 .

[14]  Zachary C. Lipton,et al.  The mythos of model interpretability , 2018, Commun. ACM.

[15]  Sung-Han Sim,et al.  Crack and Noncrack Classification from Concrete Surface Images Using Machine Learning , 2019 .

[16]  M. Z. Naser.  Machine Learning Assessment of Fiber-Reinforced Polymer-Strengthened and Reinforced Concrete Members , 2020 .

[17]  Tarek H. Almusallam,et al.  Ultimate strength prediction for RC beams externally strengthened by composite materials , 2001 .

[18]  Sergio Escalera,et al.  Explainable and Interpretable Models in Computer Vision and Machine Learning , 2018, The Springer Series on Challenges in Machine Learning.

[19]  M. Z. Naser.  Heuristic machine cognition to predict fire-induced spalling and fire resistance of concrete structures , 2019, Automation in Construction.

[20]  Gary Klein,et al.  Explaining Explanation For “Explainable Ai” , 2018, Proceedings of the Human Factors and Ergonomics Society Annual Meeting.

[21]  M. Chellapandian,et al.  Efficient near surface mounted CFRP shear strengthening of high strength prestressed concrete beams – An experimental study , 2017 .

[22]  Venkatesh Kodur,et al.  Performance comparison of fiber sheet strengthened RC beams bonded with geopolymer and epoxy resin under ambient and fire conditions , 2017 .

[23]  Jun Shang Kuang,et al.  Equivalent frame analysis for effective wall width of nonplanar beam–wall connections , 2017 .

[24]  Josep Domingo-Ferrer,et al.  Machine Learning Explainability Through Comprehensible Decision Trees , 2019, CD-MAKE.

[25]  Tim Miller,et al.  Explanation in Artificial Intelligence: Insights from the Social Sciences , 2017, Artif. Intell..

[26]  Xiaojin Zhu,et al.  Machine Teaching: An Inverse Problem to Machine Learning and an Approach Toward Optimal Education , 2015, AAAI.

[27]  Dercilio Junior Verly Lopes,et al.  Artificial neural networks modelling based on visual analysis of coated cross laminated timber (CLT) to predict color change during outdoor exposure , 2020 .

[28]  Emil Pitkin,et al.  Peeking Inside the Black Box: Visualizing Statistical Learning With Plots of Individual Conditional Expectation , 2013, 1309.6392.

[29]  Slawomir Koziel,et al.  Fast tolerance-aware design optimization of miniaturized microstrip couplers using variable-fidelity EM simulations and response features , 2019, Engineering Computations.

[30]  Maciej Laszczyk,et al.  Survey of quality measures for multi-objective optimization: Construction of complementary set of multi-objective quality measures , 2019, Swarm Evol. Comput..

[31]  Cristina Barris,et al.  Flexural response of reinforced concrete (RC) beams strengthened with near surface mounted (NSM) fibre reinforced polymer (FRP) bars , 2014 .

[32]  Oliver Kelly,et al.  Ultimate Response of RC Beams Strengthened with CFRP Plates , 2001 .

[33]  A. Nanni,et al.  BRITTLE FAILURE IN FRP PLATE AND SHEET BONDED BEAMS , 1997 .

[34]  Mohamed Osmani,et al.  Circular economy in construction: current awareness, challenges and enablers , 2017 .

[35]  Scott Lundberg,et al.  A Unified Approach to Interpreting Model Predictions , 2017, NIPS.

[36]  Amina Adadi,et al.  Peeking Inside the Black-Box: A Survey on Explainable Artificial Intelligence (XAI) , 2018, IEEE Access.

[37]  Sinan Q. Salih,et al.  Load-carrying capacity and mode failure simulation of beam-column joint connection: Application of self-tuning machine learning model , 2019, Engineering Structures.

[38]  Jiju Antony,et al.  Design of experiments for engineers and scientists , 2003 .

[39]  A. Hutchinson,et al.  CONCRETE BEAMS STRENGTHENED WITH EXTERNALLY BONDED FRP PLATES , 2001 .

[40]  Lucila Ohno-Machado,et al.  Logistic regression and artificial neural network classification models: a methodology review , 2002, J. Biomed. Informatics.

[41]  Peter Predehl,et al.  Multiband imaging at the diffraction limit using Fresnel x-ray telescopes , 2012 .

[42]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[43]  Amir Hossein Gandomi,et al.  Multi expression programming: a new approach to formulation of soil classification , 2010, Engineering with Computers.

[44]  V. Degtyarev.  Neural networks for predicting shear strength of CFS channels with slotted webs , 2020 .

[45]  Sheila Conejos,et al.  Enabling Adaptable Buildings: Results of a Preliminary Expert Survey , 2016 .

[46]  Ian H. Witten,et al.  Data mining: practical machine learning tools and techniques, 3rd Edition , 1999 .

[47]  Wang Wenwei,et al.  Experimental study and analysis of RC beams strengthened with CFRP laminates under sustaining load , 2006 .

[48]  M. Z. Naser,et al.  Leveraging artificial intelligence to assess explosive spalling in fire-exposed RC columns , 2019 .

[49]  Emadaldin Mohammadi Golafshani,et al.  Machine learning study of the mechanical properties of concretes containing waste foundry sand , 2020 .

[50]  Thomas Lengauer,et al.  Permutation importance: a corrected feature importance measure , 2010, Bioinform..

[51]  Matthias Dehmer,et al.  Explainable artificial intelligence and machine learning: A reality rooted perspective , 2020, WIREs Data Mining Knowl. Discov..

[52]  Filip Karlo Dosilovic,et al.  Explainable artificial intelligence: A survey , 2018, 2018 41st International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO).

[53]  Robert Bogue,et al.  What are the prospects for robots in the construction industry? , 2018, Ind. Robot.

[54]  Piet Demeester,et al.  A Surrogate Modeling and Adaptive Sampling Toolbox for Computer Based Design , 2010, J. Mach. Learn. Res..

[55]  Seong-Hoon Hwang,et al.  Rapid seismic damage evaluation of bridge portfolios using machine learning techniques , 2019 .

[56]  Ali Behnood,et al.  Predicting the dynamic modulus of asphalt mixture using machine learning techniques: An application of multi biogeography-based programming , 2021 .

[57]  Abdy Kermani,et al.  Machine Learning for Sustainable Structures: A Call for Data , 2019, Structures.

[58]  Conrad W. Rosenbrock,et al.  Machine-learned multi-system surrogate models for materials prediction , 2018, npj Computational Materials.

[59]  Hadi Salehi,et al.  Machine learning framework for predicting failure mode and shear capacity of ultra high performance concrete beams , 2020, Engineering Structures.

[60]  M. Z. Naser,et al.  Properties and material models for construction materials post exposure to elevated temperatures , 2020, Mechanics of Materials.

[61]  A. Kaveh,et al.  Optimum design of three-dimensional steel frames with prismatic and non-prismatic elements , 2019, Engineering with Computers.

[62]  Sebastian Raschka,et al.  Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning , 2018, ArXiv.

[63]  Yoav Freund,et al.  Experiments with a New Boosting Algorithm , 1996, ICML.

[64]  Hao Li,et al.  Visualizing the Loss Landscape of Neural Nets , 2017, NeurIPS.

[65]  Hojjat Adeli,et al.  A novel machine learning‐based algorithm to detect damage in high‐rise building structures , 2017 .

[66]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[67]  Wael W. El-Dakhakhni,et al.  Machine learning algorithms for structural performance classifications and predictions: Application to reinforced masonry shear walls , 2019 .

[68]  Tie-Yan Liu,et al.  LightGBM: A Highly Efficient Gradient Boosting Decision Tree , 2017, NIPS.

[69]  Esko Sistonen,et al.  Machine learning for durability and service-life assessment of reinforced concrete structures: Recent advances and future directions , 2017 .

[70]  Torsten Biemann,et al.  Rethinking the Gold Standard With Multi-armed Bandits: Machine Learning Allocation Algorithms for Experiments , 2019, Organizational Research Methods.

[71]  Osman E. Ozbulut,et al.  Flexural performance evaluation of NSM basalt FRP-strengthened concrete beams using digital image correlation system , 2017 .

[72]  Moncef L. Nehdi,et al.  Machine learning prediction of mechanical properties of concrete: Critical review , 2020, Construction and Building Materials.

[73]  David E. Smith,et al.  Engineering Trust in Complex Automated Systems , 2016 .

[74]  Jong-Su Jeon,et al.  Artificial neural network based multi-dimensional fragility development of skewed concrete bridge classes , 2018 .

[75]  Joaquim A. O. Barros,et al.  Behavior of RC beams flexurally strengthened with NSM CFRP laminates , 2018, Composite Structures.

[76]  Yoav Freund,et al.  A decision-theoretic generalization of on-line learning and an application to boosting , 1997, EuroCOLT.

[77]  L. Lorenzis,et al.  Near-surface mounted FRP reinforcement: An emerging technique for strengthening structures , 2007 .

[78]  Ching Chiaw Choo,et al.  Flexural Behavior of R/C Beams Strengthened with Carbon Fiber Reinforced Polymer Sheets or Fabric , 2003 .

[79]  J. Friedman.  Greedy function approximation: A gradient boosting machine. , 2001 .

[80]  T. Blanksvärd,et al.  Strengthening of RC beams using bottom and side NSM reinforcement , 2018, Composites Part B: Engineering.

[81]  F. Ceroni.  Experimental performances of RC beams strengthened with FRP materials , 2010 .

[82]  Nikhil Ketkar,et al.  Introduction to Keras , 2017 .

[83]  Mohit Bansal,et al.  Evaluating Explainable AI: Which Algorithmic Explanations Help Users Predict Model Behavior? , 2020, ACL.

[84]  Frédéric Bosché,et al.  Automated defect detection and classification in ashlar masonry walls using machine learning , 2019, Automation in Construction.

[85]  Jim Rutherford,et al.  Practical Experiment Designs for Engineers and Scientists , 2002, Technometrics.

[86]  M. Z. Naser,et al.  Concrete under fire: an assessment through intelligent pattern recognition , 2019, Engineering with Computers.

[87]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[88]  Cynthia Rudin,et al.  Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead , 2018, Nature Machine Intelligence.

[89]  Meng Zhang,et al.  High cycle fatigue life prediction of laser additive manufactured stainless steel: A machine learning approach , 2019, International Journal of Fatigue.

[90]  Bo Wang,et al.  Debonding Failures of RC Beams Strengthened with Near Surface Mounted CFRP Strips , 2006 .

[91]  Yang Liu,et al.  Fatigue Reliability Assessment of Welded Steel Bridge Decks under Stochastic Truck Loads via Machine Learning , 2017 .

[92]  Seong-Hoon Hwang,et al.  Data-driven machine-learning-based seismic failure mode identification of reinforced concrete shear walls , 2020 .

[93]  Grunde Jomaas,et al.  Testing for knowledge : maximising information obtained from fire tests by using machine learning techniques , 2019 .

[94]  Guang-Zhong Yang,et al.  XAI—Explainable artificial intelligence , 2019, Science Robotics.

[95]  Xia Hu,et al.  Techniques for interpretable machine learning , 2018, Commun. ACM.

[96]  Jacques Heyman,et al.  The Science of Structural Engineering , 1999 .

[97]  William Prager,et al.  Problems of Optimal Structural Design , 1968 .

[98]  Ke Xu,et al.  Feature extraction based on contourlet transform and its application to surface inspection of metals , 2012 .

[99]  Dong-Joong Kang,et al.  Machine learning-based imaging system for surface defect inspection , 2016, International Journal of Precision Engineering and Manufacturing-Green Technology.

[100]  Michael I. Jordan,et al.  Machine learning: Trends, perspectives, and prospects , 2015, Science.

[101]  Jamal A. Abdalla,et al.  Effects of Ratio of CFRP Plate Length to Shear Span and End Anchorage on Flexural Behavior of SCC RC Beams , 2011 .

[102]  M. Z. Naser,et al.  Finite element simulation of reinforced concrete beams externally strengthened with short-length CFRP plates , 2013 .

[103]  Charless C. Fowlkes,et al.  Do We Need More Training Data? , 2015, International Journal of Computer Vision.

[104]  D. Morgan,et al.  Opportunities and Challenges for Machine Learning in Materials Science , 2020, Annual Review of Materials Research.

[105]  Lalana Kagal,et al.  Explaining Explanations: An Overview of Interpretability of Machine Learning , 2018, 2018 IEEE 5th International Conference on Data Science and Advanced Analytics (DSAA).

[106]  M. Z. Naser.  Mechanistically Informed Machine Learning and Artificial Intelligence in Fire Engineering and Sciences , 2021, Fire Technology.