Structural dynamic response reconstruction and virtual sensing using a sequence to sequence modeling with attention mechanism

Abstract Structural condition perception is a crucial step in contemporary structural health monitoring. Sensor malfunction and under-sensing seriously hamper the performance of the structural health monitoring system. This paper proposes a novel structural dynamic response reconstruction and virtual sensing approach for structural health monitoring using a sequence-to-sequence modeling framework with a soft attention mechanism from the perspective of sequence data generation. This framework explicitly utilizes the potential spatiotemporal correlation in sequence data and promotes the efficient flow of information in the network, thereby significantly improving the reconstruction performance. In addition, a reconstruction error estimation and uncertainty quantification method based on signal complexity characterized by entropy is also developed. The effectiveness and robustness of the proposed method are verified based on the vibration signals of a footbridge measured on-site under low-amplitude ambient excitation. Finally, the applicability of this method in the scenarios of modal identification and sensor validation is demonstrated.

[1]  P. Moser,et al.  Design and deployment of a continuous monitoring system for the dowling hall footbridges , 2013, Experimental Techniques.

[2]  Jürgen Schmidhuber,et al.  LSTM: A Search Space Odyssey , 2015, IEEE Transactions on Neural Networks and Learning Systems.

[3]  Sheng Zhan,et al.  Multi-type sensor placement and response reconstruction for building structures: Experimental investigations , 2018, Earthquake Engineering and Engineering Vibration.

[4]  Jun Chen,et al.  A deep learning approach for real-time rebar counting on the construction site based on YOLOv3 detector , 2021 .

[5]  Bart De Moor,et al.  Subspace Identification for Linear Systems: Theory ― Implementation ― Applications , 2011 .

[6]  Yoshua Bengio,et al.  Show, Attend and Tell: Neural Image Caption Generation with Visual Attention , 2015, ICML.

[7]  Jiayao Chen,et al.  Quantification of water inflow in rock tunnel faces via convolutional neural network approach , 2021 .

[8]  Ye Xia,et al.  Review of Bridge Structural Health Monitoring Aided by Big Data and Artificial Intelligence: From Condition Assessment to Damage Detection , 2020 .

[9]  Sahin Yildirim,et al.  Fault detection on robot manipulators using artificial neural networks , 2011 .

[10]  Babak Moaveni,et al.  Environmental effects on the identified natural frequencies of the Dowling Hall Footbridge , 2011 .

[11]  Yan Yu,et al.  Compressive sampling–based data loss recovery for wireless sensor networks used in civil structural health monitoring , 2013 .

[12]  Gao Fan,et al.  Dynamic response reconstruction for structural health monitoring using densely connected convolutional networks , 2020, Structural Health Monitoring.

[13]  Thanh-Canh Huynh,et al.  Vision-based autonomous bolt-looseness detection method for splice connections: Design, lab-scale evaluation, and field application , 2021 .

[14]  Gaëtan Kerschen,et al.  Sensor validation using principal component analysis , 2005 .

[15]  Weiting Chen,et al.  Measuring complexity using FuzzyEn, ApEn, and SampEn. , 2009, Medical engineering & physics.

[16]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[17]  Hui Li,et al.  Compressive sensing of wireless sensors based on group sparse optimization for structural health monitoring , 2018 .

[18]  J. M. N. Silva,et al.  ON THE GENERALISATION OF THE TRANSMISSIBILITY CONCEPT , 2000 .

[19]  Kejie Jiang,et al.  A decentralized unsupervised structural condition diagnosis approach using deep auto‐encoders , 2021, Comput. Aided Civ. Infrastructure Eng..

[20]  Bin Xu,et al.  KF-Based Multiscale Response Reconstruction under Unknown Inputs with Data Fusion of Multitype Observations , 2019, Journal of Aerospace Engineering.

[21]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[22]  Yi-Qing Ni,et al.  Technology developments in structural health monitoring of large-scale bridges , 2005 .

[23]  E. Parloo,et al.  AUTONOMOUS STRUCTURAL HEALTH MONITORING—PART I: MODAL PARAMETER ESTIMATION AND TRACKING , 2002 .

[24]  Xin Zhao,et al.  Optimal multi‐type sensor placement for monitoring high‐rise buildings under bidirectional long‐period ground motions , 2020 .

[25]  Jyrki Kullaa,et al.  Detection, identification, and quantification of sensor fault in a sensor network , 2013 .

[26]  Jun Li,et al.  Structural response reconstruction with transmissibility concept in frequency domain , 2011 .

[27]  Jinping Ou,et al.  Recent progress and future trends on damage identification methods for bridge structures , 2019, Structural Control and Health Monitoring.

[28]  Jingjing He,et al.  Structural response reconstruction based on empirical mode decomposition in time domain , 2012 .

[29]  Yongchao Yang,et al.  Harnessing data structure for recovery of randomly missing structural vibration responses time history: Sparse representation versus low-rank structure , 2016 .

[30]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[31]  Jinping Ou,et al.  The state of the art in structural health monitoring of cable-stayed bridges , 2016 .

[32]  Ting Wang,et al.  Structural response reconstruction based on the modal superposition method in the presence of closely spaced modes , 2014 .

[33]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[34]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[35]  Jyrki Kullaa,et al.  Sensor validation using minimum mean square error estimation , 2010 .

[36]  Wojciech Zaremba,et al.  An Empirical Exploration of Recurrent Network Architectures , 2015, ICML.

[37]  Hui Li,et al.  Convolutional neural network‐based data anomaly detection method using multiple information for structural health monitoring , 2018, Structural Control and Health Monitoring.

[38]  Ye Xia,et al.  Lost data reconstruction for structural health monitoring using deep convolutional generative adversarial networks , 2020, Structural Health Monitoring.

[39]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[40]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[41]  Yoshua Bengio,et al.  Gradient Flow in Recurrent Nets: the Difficulty of Learning Long-Term Dependencies , 2001 .

[42]  Xiao Lu,et al.  Integrated multi‐type sensor placement and response reconstruction method for high‐rise buildings under unknown seismic loading , 2018 .

[43]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[44]  X. W. Ye,et al.  A review on deep learning-based structural health monitoring of civil infrastructures , 2019 .

[45]  Paolo Torroni,et al.  Attention in Natural Language Processing , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[46]  Kincho H. Law,et al.  Sensor data reconstruction and anomaly detection using bidirectional recurrent neural network , 2019, Smart Structures and Materials + Nondestructive Evaluation and Health Monitoring.

[47]  Jun Li,et al.  Substructural Response Reconstruction in Wavelet Domain , 2011 .

[48]  Hui Li,et al.  Compressive-sensing data reconstruction for structural health monitoring: a machine-learning approach , 2019, Structural Health Monitoring.

[49]  Rune Brincker,et al.  Modal identification of output-only systems using frequency domain decomposition , 2001 .

[50]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[51]  Yi-Qing Ni,et al.  Bayesian multi-task learning methodology for reconstruction of structural health monitoring data , 2018, Structural Health Monitoring.

[52]  Yl L. Xu,et al.  Optimal multi-type sensor placement for response and excitation reconstruction , 2016 .

[53]  Gao Fan,et al.  Lost data recovery for structural health monitoring based on convolutional neural networks , 2019, Structural Control and Health Monitoring.

[54]  Daniel C. Kammer,et al.  Estimation of structural response using remote sensor locations , 1997 .

[55]  Yaozhi Luo,et al.  Restoring method for missing data of spatial structural stress monitoring based on correlation , 2017 .

[56]  Jerome P. Lynch,et al.  Sensor data reconstruction using bidirectional recurrent neural network with application to bridge monitoring , 2019, Adv. Eng. Informatics.

[57]  Yuequan Bao,et al.  A novel distribution regression approach for data loss compensation in structural health monitoring , 2018 .

[58]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[59]  Paul J. Werbos,et al.  Backpropagation Through Time: What It Does and How to Do It , 1990, Proc. IEEE.

[60]  Erol Tutumluer,et al.  Automated crack severity level detection and classification for ballastless track slab using deep convolutional neural network , 2021 .

[61]  Songye Zhu,et al.  Multi-type sensor placement and response reconstruction for structural health monitoring of long-span suspension bridges , 2016 .

[62]  Lutz Prechelt,et al.  Early Stopping - But When? , 2012, Neural Networks: Tricks of the Trade.

[63]  B. Pompe,et al.  Permutation entropy: a natural complexity measure for time series. , 2002, Physical review letters.