3D structural vibration identification from dynamic point clouds

Abstract Video-based measurement has received increased attention for modal analysis and nondestructive evaluation, playing an important role in the development of the next-generation structural sensing technologies. As these techniques have evolved, more quantitative approaches based on computer vision techniques have emerged on full-field unsupervised structural identification, exploiting the benefits provided by the use of video cameras such as high spatial sensor density and low installation costs. More recent work has started to explore the use of laser point cloud data for 3D mapping of scenes and structures. Sensors such as LIDAR provide huge amounts of measurements at high spatial resolution from which it is possible to estimate accurate structural geometry for applications such as the generation of CAD models. Unfortunately to-date, the frame rate and depth resolution of LIDAR and other sensors capable of 3D geometry measurements has not been sufficient for measuring structural dynamics. In this paper, we introduce an approach for efficient and extremely high resolution 3D structural dynamic identification/modal analysis from point cloud data acquired using a commercial, low-cost, time-of-flight imager. Vibration mode shapes and modal coordinates are extracted from this data by creating virtual Lagrangian sensors based on the point clouds parameters. First, time-varying point cloud data are collected from a vibrating structure. Then, a mesh of virtual sensors is created based on the dynamic point cloud data for tracking the structure’s displacement over time. Next solutions to the blind source separation problem are employed to estimate high resolution 3D mode shapes, modal coordinates, and resonant frequencies. We demonstrate the potential of our proposed approach on laboratory tests and compare the results to the data collected from conventional laser displacement sensors. This technique represents an advance towards efficiently exploring the full advantages of using dynamic point cloud data for practical monitoring applications and has the potential to be extended for a wide range of 3D motion decomposition problems.

[1]  Xiaoou Tang,et al.  LiteFlowNet: A Lightweight Convolutional Neural Network for Optical Flow Estimation , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[2]  Xuetao Zhang,et al.  A method of partially overlapping point clouds registration based on differential evolution algorithm , 2018, PloS one.

[3]  Shutao Li,et al.  Proposed Methodology for Generation of Building Information Model with Laserscanning , 2008 .

[4]  Nicholette D. Palmer,et al.  Novel genetic associations for blood pressure identified via gene-alcohol interaction in up to 570K individuals across multiple ancestries , 2018, PloS one.

[5]  Susan M. Astley,et al.  Evaluation of Kinect 3D Sensor for Healthcare Imaging , 2016, Journal of medical and biological engineering.

[6]  Charles R. Farrar,et al.  Comparative study of damage identification algorithms applied to a bridge: II. Numerical study , 1998 .

[7]  D. Beskos.  Dynamic Analysis of Plates , 1991 .

[8]  X. W. Ye,et al.  Identification of structural dynamic characteristics based on machine vision technology , 2017, Measurement.

[9]  Dionisio Bernal,et al.  Flexibility-Based Approach for Damage Characterization: Benchmark Application , 2004 .

[10]  Iskender Agi,et al.  IMpixel 65nm BSI 320MHz demodulated TOF Image sensor with 3μm global shutter pixels and analog binning , 2018, 2018 IEEE International Solid - State Circuits Conference - (ISSCC).

[11]  Songye Zhu,et al.  Moving load-induced response of damaged beam and its application in damage localization , 2016 .

[12]  Yun Mook Lim,et al.  Non-contact strain measurement for laterally loaded steel plate using LiDAR point cloud displacement data , 2018, Sensors and Actuators A: Physical.

[13]  D. Boas,et al.  Laser speckle contrast imaging in biomedical optics. , 2010, Journal of biomedical optics.

[14]  Krzysztof Holak,et al.  A motion magnification application in video-based vibration measurement , 2019, Advances in Mechanism and Machine Science.

[15]  A. Zona.  Vision-Based Vibration Monitoring of Structures and Infrastructures: An Overview of Recent Applications , 2020, Infrastructures.

[16]  Erkan Beşdok,et al.  Comparison of point cloud filtering methods with data acquired by photogrammetric method and RGB-D sensors , 2021 .

[17]  Gene H. Golub,et al.  Matrix computations , 1983 .

[18]  Xiaoyi Bao,et al.  The mass loading effect on lightweight cantilever mode frequency measurement by optical fiber sensor , 2008, Photonics North.

[19]  Tania Landes,et al.  From Point Clouds to Building Information Models: 3D Semi-Automatic Reconstruction of Indoors of Existing Buildings , 2017 .

[20]  E. O'brien,et al.  The Feasibility of Using Laser Doppler Vibrometer Measurements from a Passing Vehicle for Bridge Damage Detection , 2018 .

[21]  Nicholas A. Valente,et al.  Motion magnification for optical-based structural health monitoring , 2020, Smart Structures and Materials + Nondestructive Evaluation and Health Monitoring.

[22]  Yun Mook Lim,et al.  A LiDAR Point Cloud Data-Based Method for Evaluating Strain on a Curved Steel Plate Subjected to Lateral Pressure , 2020, Sensors.

[23]  Belén Riveiro,et al.  Automated processing of large point clouds for structural health monitoring of masonry arch bridges , 2016 .

[24]  J Maeck,et al.  Dynamic Bending and Torsion Stiffness Derivation from Modal Curvatures and Torsion Rates , 1999 .

[25]  Gangolf Hirtz,et al.  Cubes3D: Neural Network based Optical Flow in Omnidirectional Image Scenes , 2018, ArXiv.

[26]  Luna Ngeljaratan,et al.  Structural health monitoring and seismic response assessment of bridge structures using target-tracking digital image correlation , 2020 .

[27]  Sérgio M. O. Tavares,et al.  Advanced design for lightweight structures: Review and prospects , 2014 .

[28]  James M. W. Brownjohn,et al.  Review of machine-vision based methodologies for displacement measurement in civil structures , 2018 .

[29]  Jacob Cano.  Point Cloud Technology for Analysis of Existing Structures , 2019 .

[30]  David J. Ewins,et al.  Modal Testing: Theory, Practice, And Application , 2000 .

[31]  Janko Slavič,et al.  Frequency domain triangulation for full-field 3D operating-deflection-shape identification , 2019, Mechanical Systems and Signal Processing.

[32]  A. Torralba,et al.  Motion magnification , 2005, SIGGRAPH 2005.

[33]  Jaehyuk Choi,et al.  Indirect Time-of-Flight Depth Sensor with Two-Step Comparison Scheme for Depth Frame Difference Detection , 2019, Sensors.

[34]  David J. Fleet,et al.  Computation of component image velocity from local phase information , 1990, International Journal of Computer Vision.

[35]  Guido M. Cortelazzo,et al.  Time-of-Flight Cameras and Microsoft Kinect™ , 2012, Springer Briefs in Electrical and Computer Engineering.

[36]  B. Feeny,et al.  On the physical interpretation of proper orthogonal modes in vibrations , 1998 .

[37]  Marco Amati,et al.  Individual Tree Canopy Parameters Estimation Using UAV-Based Photogrammetric and LiDAR Point Clouds in an Urban Park , 2021, Remote. Sens..

[38]  Jean-Louis Batoz,et al.  From Point Cloud Data to Structural Analysis Through a Geometrical hBIM-Oriented Model , 2019, ACM Journal on Computing and Cultural Heritage.

[39]  Alessandro Sabato,et al.  Digital Image Correlation Techniques for NDE and SHM , 2019, Handbook of Advanced Nondestructive Evaluation.

[40]  Frédo Durand,et al.  Modal identification of simple structures with high-speed video using motion magnification , 2015 .

[41]  John E. Mottershead,et al.  Finite Element Model Updating in Structural Dynamics , 1995 .

[42]  Charles R. Farrar,et al.  Blind identification of full-field vibration modes from video measurements with phase-based video motion magnification , 2017 .

[43]  Raffaele Zinno,et al.  A new low-cost displacements monitoring system based on Kinect sensor , 2015 .

[44]  Livio Pinto,et al.  Calibration of Kinect for Xbox One and Comparison between the Two Generations of Microsoft Sensors , 2015, Sensors.

[45]  Alessandro Sabato,et al.  Feasibility of using digital image correlation for unmanned aerial vehicle structural health monitoring of bridges , 2018 .

[46]  Tadeusz Uhl,et al.  A Review of Laser Doppler Vibrometry for Structural Health Monitoring Applications , 2012 .

[47]  Javad Baqersad,et al.  A multi-view optical technique to obtain mode shapes of structures , 2018, Measurement.

[48]  Ángel J. Molina-Viedma,et al.  3D mode shapes characterisation using phase-based motion magnification in large structures using stereoscopic DIC , 2018, Mechanical Systems and Signal Processing.

[49]  Marc M. Van Hulle,et al.  A phase-based approach to the estimation of the optical flow field using spatial filtering , 2002, IEEE Trans. Neural Networks.

[50]  Lee Sze Wei,et al.  Comparative Study of Intel R200, Kinect v2, and Primesense RGB-D Sensors Performance Outdoors , 2019, IEEE Sensors Journal.

[51]  Frédo Durand,et al.  Eulerian video magnification for revealing subtle changes in the world , 2012, ACM Trans. Graph..

[52]  Maria Q. Feng,et al.  Identification of structural stiffness and excitation forces in time domain using noncontact vision-based displacement measurement , 2017 .

[53]  Hong-Wei Ma,et al.  Physical Interpretation of Principal Component Analysis for Structural Dynamics Through String Vibration , 2019, International Journal of Structural Stability and Dynamics.

[54]  James V. Stone.  Blind Source Separation Using Temporal Predictability , 2001, Neural Computation.

[55]  Satish Nagarajaiah,et al.  Structural damage identification via a combination of blind feature extraction and sparse representation classification , 2014 .

[56]  Maria Q. Feng,et al.  A Vision-Based Sensor for Noncontact Structural Displacement Measurement , 2015, Sensors.

[57]  David J. Ewins,et al.  MODAL TESTING USING A SCANNING LASER DOPPLER VIBROMETER , 1999 .

[58]  Mustafa Gul,et al.  Conceptual damage-sensitive features for structural health monitoring: Laboratory and field demonstrations , 2008 .

[59]  Maria Q. Feng,et al.  Computer vision for SHM of civil infrastructure: From dynamic response measurement to damage detection – A review , 2018 .

[60]  Tuba KURBAN,et al.  3 Dimensional Point Cloud Filtering Using Differential Evolution Algorithm , 2019, 2019 IEEE Jordan International Joint Conference on Electrical Engineering and Information Technology (JEEIT).

[61]  Ahmet E. Aktan,et al.  Bridge-condition assessment by modal flexibility , 1994 .

[62]  Ingo Neumann,et al.  The Benefit of 3D Laser Scanning Technology in the Generation and Calibration of FEM Models for Health Assessment of Concrete Structures , 2014, Sensors.

[63]  Ramón Doallo,et al.  Big Data Geospatial Processing for Massive Aerial LiDAR Datasets , 2020, Remote. Sens..

[64]  On the orthogonality of natural modes of vibration , 2004 .

[65]  Wei Fan,et al.  Vibration-based Damage Identification Methods: A Review and Comparative Study , 2011 .

[66]  Harith T. Al-Jumaily,et al.  Big-Data Approach for Three-Dimensional Building Extraction from Aerial Laser Scanning , 2016, J. Comput. Civ. Eng..

[67]  M. Géradin,et al.  Mechanical Vibrations: Theory and Application to Structural Dynamics , 1994 .

[68]  Mauro R. Ruggeri,et al.  Isometry-invariant Matching of Point Set Surfaces , 2008, 3DOR@Eurographics.

[69]  Hao Yang,et al.  Network method for deformation analysis of three-dimensional point cloud with terrestrial laser scanning sensor , 2018, Int. J. Distributed Sens. Networks.

[70]  Nicolas David,et al.  Towards 3D lidar point cloud registration improvement using optimal neighborhood knowledge , 2013 .

[71]  David Mascareñas,et al.  Spatio-temporal decomposition of 2D travelling waves from video measurements , 2020 .

[72]  Yongchao Yang,et al.  Blind modal identification of output‐only structures in time‐domain based on complexity pursuit , 2013 .

[73]  Ling Shao,et al.  Enhanced Computer Vision With Microsoft Kinect Sensor: A Review , 2013, IEEE Transactions on Cybernetics.

[74]  Giuseppina Vacca,et al.  Structure from Motion Point Clouds for Structural Monitoring , 2019, Remote. Sens..

[75]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..