Computer vision based target-free 3D vibration displacement measurement of structures

Abstract Vibration displacement response is widely used in the field of structural health monitoring (SHM) to monitor the health condition of civil engineering structures. Traditionally, the vibration displacement response is acquired by physical sensors, such as linear variable differential transformers (LVDT) and laser displacement sensors (LDS), or more commonly by accelerometers to measure accelerations instead of directly measuring the displacement due to the fact that direct displacement measurement requires a fixed platform to install the sensors. Recently, to overcome the difficulties in direct displacement measurement, computer vision based methods have become a research hotspot. This paper proposes an advanced binocular vision system for target-free full-field three-dimensional (3D) vibration displacement measurement of civil engineering structures. The state-of-the-art key point detection and matching algorithm based on deep learning is employed to achieve target-free measurement, which greatly improves the quantity and quality of matching natural key points with low contrast. The performance of the proposed vision based approach for 3D vibration displacement measurement is evaluated through an experimental test on a steel cantilever beam in the laboratory. The obtained vibration displacement responses obtained from the proposed approach are compared with those measured by LVDTs and LDSs. The results demonstrate that the displacement responses obtained from the proposed vision based approach are accurate compared with the traditional sensors, while the proposed approach is more cost effective and much easier to achieve accurate 3D vibration displacement measurement.

[1]  Jukka-Pekka Onnela,et al.  Community Structure in Time-Dependent, Multiscale, and Multiplex Networks , 2009, Science.

[2]  R. Hartley.  Triangulation, Computer Vision and Image Understanding , 1997 .

[3]  Joseph Morlier,et al.  Virtual Vibration Measurement Using KLT Motion Tracking Algorithm , 2010 .

[4]  Ayan Sadhu,et al.  A literature review of next‐generation smart sensing technology in structural health monitoring , 2019, Structural Control and Health Monitoring.

[5]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[6]  Jiri Matas,et al.  Forward-Backward Error: Automatic Detection of Tracking Failures , 2010, 2010 20th International Conference on Pattern Recognition.

[7]  David J. Fleet,et al.  Performance of optical flow techniques , 1994, International Journal of Computer Vision.

[8]  Marco Cuturi,et al.  Sinkhorn Distances: Lightspeed Computation of Optimal Transport , 2013, NIPS.

[9]  C. Tomasi.  Detection and Tracking of Point Features , 1991 .

[10]  Jung-Hoon Kim,et al.  3D displacement measurement model for health monitoring of structures using a motion capture system , 2015 .

[11]  Adrien Bartoli,et al.  KAZE Features , 2012, ECCV.

[12]  Maria Q. Feng,et al.  Vision‐based multipoint displacement measurement for structural health monitoring , 2016 .

[13]  Chuan-Zhi Dong,et al.  Structural displacement monitoring using deep learning-based full field optical flow methods , 2020, Structure and Infrastructure Engineering.

[14]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[15]  Roland Siegwart,et al.  BRISK: Binary Robust invariant scalable keypoints , 2011, 2011 International Conference on Computer Vision.

[16]  Philip H. S. Torr,et al.  The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix , 1997, International Journal of Computer Vision.

[17]  Tomasz Malisiewicz,et al.  SuperGlue: Learning Feature Matching With Graph Neural Networks , 2019, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Jean-José Orteu,et al.  3-D computer vision in experimental mechanics , 2007 .

[19]  Chih-Chen Chang,et al.  Nontarget Image-Based Technique for Small Cable Vibration Measurement , 2008 .

[20]  Marc Barthelemy,et al.  Growing multiplex networks , 2013, Physical review letters.

[21]  Torsten Sattler,et al.  D2-Net: A Trainable CNN for Joint Description and Detection of Local Features , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[22]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[23]  Vincent Lepetit,et al.  Learning to Find Good Correspondences , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[24]  M. Chun,et al.  Contextual cueing of visual attention , 2022 .

[25]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[26]  Tomasz Malisiewicz,et al.  SuperPoint: Self-Supervised Interest Point Detection and Description , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[27]  Maria Q. Feng,et al.  Cost‐effective vision‐based system for monitoring dynamic response of civil engineering structures , 2010 .

[28]  Fanis Moschas,et al.  GPS/RTS data fusion to overcome signal deficiencies in certain bridge dynamic monitoring projects , 2013 .

[29]  Mani Golparvar-Fard,et al.  Target‐free approach for vision‐based structural system identification using consumer‐grade cameras , 2016 .

[30]  Kyle T. Wieghaus,et al.  Target-less computer vision for traffic signal structure vibration studies , 2015 .

[31]  Hong Hao,et al.  Target-free vision-based technique for vibration measurements of structures subjected to out-of-plane movements , 2019, Engineering Structures.

[32]  Emanuele Zappa,et al.  Vibration Monitoring of Multiple Bridge Points by Means of a Unique Vision-Based Measuring System , 2014 .

[33]  Xiaohua Tong,et al.  DEVELOPMENT OF A HIGH-SPEED VIDEOGRAMMETRIC MEASUREMENT SYSTEM WITH APPLICATION IN LARGE-SCALE SHAKING TABLE TEST , 2019 .

[34]  Luc Van Gool,et al.  SURF: Speeded Up Robust Features , 2006, ECCV.

[35]  Andrea Vedaldi,et al.  HPatches: A Benchmark and Evaluation of Handcrafted and Learned Local Descriptors , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[36]  Maria Q. Feng,et al.  Computer vision for SHM of civil infrastructure: From dynamic response measurement to damage detection – A review , 2018 .

[37]  Uvais Qidwai,et al.  3D dynamic displacement-field measurement for structural health monitoring using inexpensive RGB-D based sensor , 2017 .

[38]  Silvio Savarese,et al.  Universal Correspondence Network , 2016, NIPS.

[39]  Matthias Nießner,et al.  ScanNet: Richly-Annotated 3D Reconstructions of Indoor Scenes , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[40]  Daniel Rueckert,et al.  Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Sunglok Choi,et al.  Performance Evaluation of RANSAC Family , 2009, BMVC.

[42]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[43]  Lei Yang,et al.  Identification of dynamic displacements and modal frequencies of a medium-span suspension bridge using multimode GNSS processing , 2014 .

[44]  Claudio Amadio,et al.  Vibration Analysis and Dynamic Characterization of Structural Glass Elements with Different Restraints Based on Operational Modal Analysis , 2019, Buildings.

[45]  Pierre Vandergheynst,et al.  FREAK: Fast Retina Keypoint , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Jan-Michael Frahm,et al.  USAC: A Universal Framework for Random Sample Consensus , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Richard Sinkhorn,et al.  Concerning nonnegative matrices and doubly stochastic matrices , 1967 .

[48]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[49]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[50]  Maria Q. Feng,et al.  Experimental validation of cost-effective vision-based structural health monitoring , 2017 .

[51]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[53]  Rui Calçada,et al.  Non-contact measurement of the dynamic displacement of railway bridges using an advanced video-based system , 2014 .

[54]  Long Quan,et al.  Learning Two-View Correspondences and Geometry Using Order-Aware Network , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[55]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[56]  Bin Xu,et al.  Global Navigation Satellite System‐based positioning technology for structural health monitoring: a review , 2019, Structural Control and Health Monitoring.