3D dynamic displacement-field measurement for structural health monitoring using inexpensive RGB-D based sensor

The advent of inexpensive digital cameras with depth sensing capabilities (RGB-D cameras) has opened the door to numerous useful applications that need quantitative measures of dynamic fields whose simultaneous time history quantification (at many points as dictated by the resolution of the camera) provides capabilities that were previously accessible only through expensive sensors (e.g., laser scanners). This paper presents a comprehensive experimental and computational study to evaluate the performance envelope of a representative RGB-D sensor (the first generation of Kinect sensor) with the aim of assessing its suitability for the class of problems encountered in the structural dynamics field, where reasonably accurate information of evolving displacement fields (as opposed to few discrete locations) that have simultaneous dynamic planar translational motion with significant rotational (torsional) components. This study investigated the influence of key system parameters of concern in selecting an appropriate sensor for such structural dynamic applications, such as amplitude range, spectral content of the dynamic displacements, location and orientation of sensors relative to target structure, fusing of measurements from multiple sensors, sensor noise effects, rolling-shutter effects, etc. The calibration results show that if the observed displacement field generates discrete (pixel) sensor measurements with sufficient resolution (observed displacements more than 10 mm) beyond the sensor noise floor, then the subject sensors can typically provide reasonable accuracy for transnational motion (about 5%) when the frequency range of the evolving field is within about 10 Hz. However, the expected error for torsional measurements is around 6% for static motion and 10% for dynamic rotation for measurements greater than 5°.

[1]  Simon Weidert,et al.  Kinect for interactive AR anatomy learning , 2013, 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR).

[2]  Luigi Barazzetti,et al.  Photogrammetric techniques for monitoring tunnel deformation , 2014, Earth Science Informatics.

[3]  Andreas J. Kappos,et al.  Semi-Active Control Systems in Bridge Engineering: A Review of the Current State of Practice , 2016 .

[4]  Derek D. Lichti,et al.  Vertical Dynamic Deflection Measurement in Concrete Beams with the Microsoft Kinect , 2014, Sensors.

[5]  Philippe Martinet,et al.  Simultaneous Object Pose and Velocity Computation Using a Single View from a Rolling Shutter Camera , 2006, ECCV.

[6]  Daniel O. Sales,et al.  Mobile Robots Navigation in Indoor Environments Using Kinect Sensor , 2012, 2012 Second Brazilian Conference on Critical Embedded Systems.

[7]  Kiyoung Kim,et al.  Dynamic displacement measurement of a vibratory object using a terrestrial laser scanner , 2015 .

[8]  Barry L. Stann,et al.  Low-cost compact MEMS scanning ladar system for robotic applications , 2012, Other Conferences.

[9]  Kevin A. Snook,et al.  縦方向電界場中で曲げたPIN-PMN-PT単結晶の強度 , 2011 .

[10]  Zhishen Wu,et al.  Assessment of Vibration-based Damage Identification Methods Using Displacement and Distributed Strain Measurements , 2009 .

[11]  Jong-Han Lee,et al.  An advanced vision-based system for real-time displacement measurement of high-rise buildings , 2012 .

[12]  Andreas Kolb,et al.  Kinect range sensing: Structured-light versus Time-of-Flight Kinect , 2015, Comput. Vis. Image Underst..

[13]  B. Watanapa,et al.  Human gesture recognition using Kinect camera , 2012, 2012 Ninth International Conference on Computer Science and Software Engineering (JCSSE).

[14]  Roberto Horowitz,et al.  Dynamics and control of a MEMS angle measuring gyroscope , 2008 .

[15]  Jin-Bong Kim,et al.  Non-baseline Damage Detection Based on the Deviation of Displacement Mode Shape Data , 2013 .

[16]  Gordon Petrie,et al.  Terrestrial Laser Scanners , 2017 .

[17]  Stefan Hurlebaus,et al.  Summary Review of GPS Technology for Structural Health Monitoring , 2013 .

[18]  Kyle T. Wieghaus,et al.  Target-less computer vision for traffic signal structure vibration studies , 2015 .

[19]  Junsong Yuan,et al.  Robust Part-Based Hand Gesture Recognition Using Kinect Sensor , 2013, IEEE Transactions on Multimedia.

[20]  Hyun Myung,et al.  Experimental Validation of Visually Servoed Paired Structured Light System (ViSP) for Structural Displacement Monitoring , 2014, IEEE/ASME Transactions on Mechatronics.

[21]  Xingmin Hou,et al.  Using Inclinometers to Measure Bridge Deflection , 2005 .

[22]  John E. Mottershead,et al.  Finite Element Model Updating in Structural Dynamics , 1995 .

[23]  Hyun Myung,et al.  A paired visual servoing system for 6-DOF displacement measurement of structures , 2011 .

[24]  Maria Q. Feng,et al.  Vision‐based multipoint displacement measurement for structural health monitoring , 2016 .

[25]  Mohammad R. Jahanshahi,et al.  Color and depth data fusion using an RGB‐D sensor for inexpensive and contactless dynamic displacement‐field measurement , 2018 .

[26]  Partha Pratim Das,et al.  Characterizations of Noise in Kinect Depth Images: A Review , 2014, IEEE Sensors Journal.

[27]  Burcin Becerik-Gerber,et al.  Inexpensive Multimodal Sensor Fusion System for Autonomous Data Acquisition of Road Surface Conditions , 2016, IEEE Sensors Journal.

[28]  Simon A. Carn,et al.  An innovative application of the Kinect in Earth sciences: quantifying deformation in analogue modelling of volcanoes , 2014 .

[29]  Sung-Han Sim,et al.  Development of a Wireless Displacement Measurement System Using Acceleration Responses , 2013, Sensors.

[30]  Sander Oude Elberink,et al.  Accuracy and Resolution of Kinect Depth Data for Indoor Mapping Applications , 2012, Sensors.

[31]  Hoon Sohn,et al.  Autonomous dynamic displacement estimation from data fusion of acceleration and intermittent displacement measurements , 2014 .

[32]  P. Elangovan,et al.  Assessment of Microsoft Kinect technology (Kinect for Xbox and Kinect for windows) for patient monitoring during external beam radiotherapy , 2013, 2013 IEEE Nuclear Science Symposium and Medical Imaging Conference (2013 NSS/MIC).

[33]  Hojjat Adeli,et al.  A New Approach for Health Monitoring of Structures: Terrestrial Laser Scanning , 2007, Comput. Aided Civ. Infrastructure Eng..

[34]  Charles R. Farrar,et al.  Blind identification of full-field vibration modes of output-only structures from uniformly-sampled, possibly temporally-aliased (sub-Nyquist), video measurements , 2017 .

[35]  Hsi-Yung Feng,et al.  On the normal vector estimation for point cloud data from smooth surfaces , 2005, Comput. Aided Des..

[36]  Sungsu Park Adaptive Control of a Vibratory Angle Measuring Gyroscope , 2010, Sensors.

[37]  Inmaculada Coma,et al.  Augmented Mirror: Interactive Augmented Reality System Based on Kinect , 2011, INTERACT.

[38]  Gary R. Bradski,et al.  Learning OpenCV - computer vision with the OpenCV library: software that sees , 2008 .

[39]  Ramesh Raskar,et al.  Polarized 3D: High-Quality Depth Sensing with Polarization Cues , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[40]  Falko Kuester,et al.  Terrestrial Laser Scanning-Based Structural Damage Assessment , 2010, J. Comput. Civ. Eng..

[41]  Ting-Hua Yi,et al.  Recent research and applications of GPS‐based monitoring technology for high‐rise structures , 2013 .

[42]  Frédo Durand,et al.  Modal identification of simple structures with high-speed video using motion magnification , 2015 .

[43]  T. Kijewski-Correa,et al.  Monitoring the wind-induced response of tall buildings: GPS performance and the issue of multipath effects , 2007 .

[44]  Kenneth R. White,et al.  Close-range photogrammetry applications in bridge measurement: Literature review , 2008 .

[45]  Mohit Gupta,et al.  MC3D: Motion Contrast 3D Scanning , 2015, 2015 IEEE International Conference on Computational Photography (ICCP).

[46]  Jidong Huang,et al.  Study on the use of Microsoft Kinect for robotics applications , 2012, Proceedings of the 2012 IEEE/ION Position, Location and Navigation Symposium.

[47]  Sami F. Masri,et al.  A vision-based approach for the direct measurement of displacements in vibrating systems , 2003 .

[48]  Jung-Hoon Kim,et al.  3D displacement measurement model for health monitoring of structures using a motion capture system , 2015 .

[49]  Alberto Olivares,et al.  High-efficiency low-cost accelerometer-aided gyroscope calibration , 2009, 2009 International Conference on Test and Measurement.

[50]  Charles R. Farrar,et al.  Damage identification and health monitoring of structural and mechanical systems from changes in their vibration characteristics: A literature review , 1996 .

[51]  Francisco José Madrid-Cuevas,et al.  Automatic generation and detection of highly reliable fiducial markers under occlusion , 2014, Pattern Recognit..

[52]  Lijun Wu,et al.  Dynamic testing of a laboratory model via vision-based sensing , 2014 .

[53]  Jerome P. Lynch,et al.  A summary review of wireless sensors and sensor networks for structural health monitoring , 2006 .

[54]  Andrea Fossati,et al.  Consumer Depth Cameras for Computer Vision , 2013, Advances in Computer Vision and Pattern Recognition.

[55]  A. Habib,et al.  Dynamic beam deformation measurements with off-the-shelf digital cameras , 2013 .

[56]  Charles R. Farrar,et al.  Damage Detection and Evaluation II , 1999 .

[57]  Charles R. Farrar,et al.  An overview of modal-based damage identification methods , 1997 .

[58]  Ramesh Raskar,et al.  3D Depth Cameras in Vision: Benefits and Limitations of the Hardware , 2014 .

[59]  Chin-Hsiung Loh,et al.  Monitoring of long‐term static deformation data of Fei‐Tsui arch dam using artificial neural network‐based approaches , 2013 .

[60]  Andrew Weinfield Gesture Recognition Using Microsoft’s Kinect v2 , 2014 .

[61]  Jean-Yves Bouguet,et al.  Camera calibration toolbox for matlab , 2001 .

[62]  Fatemeh Tahavori,et al.  A quantitative assessment of using the Kinect for Xbox 360 for respiratory surface motion tracking , 2012, Medical Imaging.

[63]  Saso Tomazic,et al.  Angle Estimation of Simultaneous Orthogonal Rotations from 3D Gyroscope Measurements , 2011, Sensors.

[64]  Mehmet Çelebi,et al.  GPS in Pioneering Dynamic Monitoring of Long-Period Structures , 2002 .

[65]  Guido De Roeck,et al.  Dealing with uncertainty in model updating for damage assessment: A review , 2015 .

[66]  Adrian Kaehler,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[67]  Roger Ghanem,et al.  A novel approach for the structural identification and monitoring of a full-scale 17-story building based on ambient vibration measurements , 2008 .

[68]  Peter Avitabile,et al.  Extracting full-field dynamic strain on a wind turbine rotor subjected to arbitrary excitations using 3D point tracking and a modal expansion technique , 2015 .

[69]  A. W. Smyth The potential of GPS and other displacement sensing for enhancing acceleration sensor monitoring array data by solving low frequency integration problems : Andrew W. Smyth , 2004 .

[70]  Ahmet E. Aktan,et al.  ISSUES IN INFRASTRUCTURE HEALTH MONITORING FOR MANAGEMENT , 2000 .

[71]  Derek D. Lichti,et al.  Modeling Terrestrial Laser Scanner Data for Precise Structural Deformation Measurement , 2007 .

[72]  Mohammad R. Jahanshahi,et al.  Color and depth data fusion using an RGB‐D sensor for inexpensive and contactless dynamic displacement‐field measurement , 2017 .

[73]  Hoon Sohn,et al.  A review of structural health monitoring literature 1996-2001 , 2002 .

[74]  Ivan Bartoli,et al.  Full‐field deformation measurements during seismic loading of masonry buildings , 2017 .

[75]  Charles R. Farrar,et al.  Blind identification of full-field vibration modes from video measurements with phase-based video motion magnification , 2017 .

[76]  Hyun Myung,et al.  Vision-based displacement measurement method for high-rise building structures using partitioning approach , 2010 .

[77]  Piotr Górski,et al.  Monitoring horizontal displacements in a vertical profile of a tall industrial chimney using Global Positioning System technology for detecting dynamic characteristics , 2015 .

[78]  T. Chmielewski,et al.  Application of GPS technology to measurements of displacements of high-rise structures due to weak winds , 2002 .