Review of machine-vision based methodologies for displacement measurement in civil structures

Vision-based systems are promising tools for displacement measurement in civil structures, possessing advantages over traditional displacement sensors in instrumentation cost, installation efforts and measurement capacity in terms of frequency range and spatial resolution. Approximately one hundred papers to date have appeared on this subject, investigating topics like system development and improvement, the viability on field applications and the potential for structural condition assessment. The main contribution of this paper is to present a literature review of vision-based displacement measurement, from the perspectives of methodologies and applications. Video-processing procedures in this paper are summarised as a three-component framework: camera calibration, target tracking and structural displacement calculation. Methods for each component are presented in principle, with discussions about the relative advantages and limitations. Applications in the two most active fields, bridge deformation and cable vibration measurement, are examined followed by a summary of field challenges observed in monitoring tests. Important gaps requiring further investigation are presented, e.g. robust tracking methods, non-contact sensing and measurement accuracy evaluation in field conditions.

[1]  Niannian Wang,et al.  Experimental Verification for Cable Force Estimation Using Handheld Shooting of Smartphones , 2017, J. Sensors.

[2]  Joan R. Casas,et al.  Bridge Damage Detection Based on Vibration Data: Past and New Developments , 2017, Front. Built Environ..

[3]  Andrew W. Smyth,et al.  Monocular Computer Vision Method for the Experimental Study of Three-Dimensional Rocking Motion , 2016 .

[4]  Tan Liu,et al.  A Review of Machine Vision-Based Structural Health Monitoring: Methodologies and Applications , 2016, J. Sensors.

[5]  Nam-Sik Kim,et al.  Vision-based monitoring system for evaluating cable tensile forces on a cable-stayed bridge , 2013 .

[6]  Jorge Batista,et al.  A vision-based system for measuring the displacements of large structures: Simultaneous adaptive calibration and full motion estimation , 2016 .

[7]  Michael A. Sutton,et al.  Advances in light microscope stereo vision , 2004 .

[8]  R. Dinesh,et al.  Non-parametric adaptive region of support useful for corner detection: a novel approach , 2004, Pattern Recognit..

[9]  Donghyun Kim,et al.  A Target-Less Vision-Based Displacement Sensor Based on Image Convex Hull Optimization for Measuring the Dynamic Response of Building Structures , 2016, Sensors.

[10]  Ali Khaloo,et al.  Pixel‐wise structural motion tracking from rectified repurposed videos , 2017 .

[11]  Richard Szeliski,et al.  Computer Vision - Algorithms and Applications , 2011, Texts in Computer Science.

[12]  Young-Soo Park,et al.  A Synchronized Multipoint Vision-Based System for Displacement Measurement of Civil Infrastructures , 2012, TheScientificWorldJournal.

[13]  L. Lages Martins,et al.  Structural observation of long-span suspension bridges for safety assessment: implementation of an optical displacement measurement system , 2015 .

[14]  C. Fuggini,et al.  Engineering vibration monitoring by GPS: long duration records , 2009 .

[15]  Simon Baker,et al.  Lucas-Kanade 20 Years On: A Unifying Framework , 2004, International Journal of Computer Vision.

[16]  Joav Merrick,et al.  Effectiveness of the Project P.A.T.H.S. in Hong Kong: Evaluation Based on Different Strategies and Different Studies over Time , 2012, TheScientificWorldJournal.

[17]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[18]  Nam-Sik Kim,et al.  Dynamic characteristics of suspension bridge hanger cables using digital image processing , 2013 .

[19]  Peter Avitabile,et al.  Photogrammetry and optical methods in structural dynamics – A review , 2017 .

[20]  Julian Padget,et al.  Virtual visual sensors and their application in structural health monitoring , 2014 .

[21]  Anand Asundi,et al.  Two-dimensional digital image correlation for in-plane displacement and strain measurement: a review , 2009 .

[22]  P. Rousseeuw.  Least Median of Squares Regression , 1984 .

[23]  Elsa Caetano,et al.  A vision system for vibration monitoring of civil engineering structures , 2011 .

[24]  Oral Büyüköztürk,et al.  Output-only computer vision based damage detection using phase-based optical flow and unscented Kalman filters , 2017 .

[25]  Gwolong Lai,et al.  Application of digital photogrammetry techniques in identifying the mode shape ratios of stay cables with multiple camcorders , 2015 .

[26]  Hisao Kikuta,et al.  Bridge deflection measurement using digital image correlation , 2007 .

[27]  Pierre Vandergheynst,et al.  FREAK: Fast Retina Keypoint , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Chih-Chen Chang,et al.  Nontarget Stereo Vision Technique for Spatiotemporal Response Measurement of Line-Like Structures , 2008 .

[29]  S. Guillaume,et al.  Measuring sub-mm structural displacements using QDaedalus: a digital clip-on measuring system developed for total stations , 2015 .

[30]  Hak Kyeong Kim,et al.  A Vision System for Identifying Structural Vibration in Civil Engineering Constructions , 2006, 2006 SICE-ICASE International Joint Conference.

[31]  Piotr Olaszek,et al.  Investigation of the dynamic characteristic of bridge structures using a computer vision method , 1999 .

[32]  Chih-Chen Chang,et al.  Three-Dimensional Structural Translation and Rotation Measurement Using Monocular Videogrammetry , 2010 .

[33]  Vincent Lepetit,et al.  BRIEF: Binary Robust Independent Elementary Features , 2010, ECCV.

[34]  Maria Q. Feng,et al.  A Vision-Based Sensor for Noncontact Structural Displacement Measurement , 2015, Sensors.

[35]  Nam-Sik Kim,et al.  Multi-point Displacement Response Measurement of Civil Infrastructures Using Digital Image Processing , 2011 .

[36]  Adam Gąska,et al.  Development of a vision based deflection measurement system and its accuracy assessment , 2013 .

[37]  Elsa Caetano,et al.  Application of a vision system to the monitoring of cable structures , 2007 .

[38]  Jong-Jae Lee,et al.  A vision-based system for remote sensing of bridge displacement , 2006 .

[39]  José Manuel Rebordão,et al.  Thermal Influence on Long-Distance Optical Measurement of Suspension Bridge Displacement , 2014 .

[40]  Eugene J. O'Brien,et al.  Contactless Bridge Weigh-in-Motion , 2016 .

[41]  Maria Q. Feng,et al.  Experimental validation of cost-effective vision-based structural health monitoring , 2017 .

[42]  Marc M. Van Hulle,et al.  A phase-based approach to the estimation of the optical flow field using spatial filtering , 2002, IEEE Trans. Neural Networks.

[43]  Chih-Chen Chang,et al.  Flexible Videogrammetric Technique for Three-Dimensional Structural Vibration Measurement , 2007 .

[44]  X. W. Ye,et al.  A vision-based system for dynamic displacement measurement of long-span bridges : algorithm and verification , 2013 .

[45]  Hubert W. Schreier,et al.  Image Correlation for Shape, Motion and Deformation Measurements: Basic Concepts,Theory and Applications , 2009 .

[46]  Jin-Hak Yi,et al.  Field evaluation of optical-based three-dimensional dynamic motion measurement system with multiple targets for a floating structure , 2013 .

[47]  Jae Hong Min,et al.  Real-time image processing for non-contact monitoring of dynamic displacements using smartphone technologies , 2016, SPIE Smart Structures and Materials + Nondestructive Evaluation and Health Monitoring.

[48]  Werner Lienhart,et al.  Monitoring of Civil Engineering Structures using a State-of-the-art Image Assisted Total Station , 2015 .

[49]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[50]  David J. Fleet,et al.  Computation of component image velocity from local phase information , 1990, International Journal of Computer Vision.

[51]  Chih-Chen Chang,et al.  Nontarget Image-Based Technique for Small Cable Vibration Measurement , 2008 .

[52]  Maria Q. Feng,et al.  Cable tension force estimate using novel noncontact vision-based sensor , 2017 .

[53]  Xie Huimin,et al.  Performance of sub-pixel registration algorithms in digital image correlation , 2006 .

[54]  Maria Q. Feng,et al.  Vision-Based Displacement Sensor for Monitoring Dynamic Response Using Robust Object Search Algorithm , 2013 .

[55]  Carlo Tomasi,et al.  Good features to track , 1994, 1994 Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[56]  Gary R. Bradski,et al.  Learning OpenCV 3: Computer Vision in C++ with the OpenCV Library , 2016 .

[57]  Zhengyou Zhang,et al.  A Flexible New Technique for Camera Calibration , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[58]  Steven S. Beauchemin,et al.  The computation of optical flow , 1995, CSUR.

[59]  Mani Golparvar-Fard,et al.  Target‐free approach for vision‐based structural system identification using consumer‐grade cameras , 2016 .

[60]  Michael Brady,et al.  On the Choice of Band-Pass Quadrature Filters , 2004, Journal of Mathematical Imaging and Vision.

[61]  Emanuele Zappa,et al.  Vibration Monitoring of Multiple Bridge Points by Means of a Unique Vision-Based Measuring System , 2014 .

[62]  Z. L. Xie,et al.  Temperature effects on vision measurement system in long-term continuous monitoring of displacement , 2017 .

[63]  Jin-Hee Ahn,et al.  Structural dynamic displacement vision system using digital image processing , 2011 .

[64]  Paul Reynolds.  Experimental Techniques: Developments, Applications and Tutorials in Experimental Mechanics Techniques , 2015, Experimental Techniques.

[65]  James M. W. Brownjohn,et al.  Measurements of static and dynamic displacement from visual monitoring of the Humber Bridge , 1993 .

[66]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[67]  Sami F. Masri,et al.  A vision-based approach for the direct measurement of displacements in vibrating systems , 2003 .

[68]  Maria Q. Feng,et al.  Model Updating of Railway Bridge Using In Situ Dynamic Displacement Measurement under Trainloads , 2015 .

[69]  Frédo Durand,et al.  Modal identification of simple structures with high-speed video using motion magnification , 2015 .

[70]  Hyo Seon Park,et al.  Vision-based system identification technique for building structures using a motion capture system , 2015 .

[71]  Werner Lienhart,et al.  Image-based dynamic deformation monitoring of civil engineering structures from long ranges , 2015, Electronic Imaging.

[72]  Stathis C. Stiros,et al.  Monitoring dynamic and quasi-static deformations of large flexible engineering structures with GPS: Accuracy, limitations and promises , 2006 .

[73]  Michael J. Black,et al.  Secrets of optical flow estimation and their principles , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[74]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[75]  Lijun Wu,et al.  Dynamic testing of a laboratory model via vision-based sensing , 2014 .

[76]  Chin-Ling Chen,et al.  An Intelligent and Secure Health Monitoring Scheme Using IoT Sensor Based on Cloud Computing , 2017, J. Sensors.

[77]  Satoru Yoneyama,et al.  Bridge Deflection Measurement Using Digital Image Correlation with Camera Movement Correction , 2012 .

[78]  Frédo Durand,et al.  Video Camera–Based Vibration Measurement for Civil Infrastructure Applications , 2017 .

[79]  Jie Chen,et al.  The Improvement of DS Evidence Theory and Its Application in IR/MMW Target Recognition , 2015, J. Sensors.

[80]  Charles R. Farrar,et al.  Blind identification of full-field vibration modes from video measurements with phase-based video motion magnification , 2017 .

[81]  Yan Xu,et al.  Vision-Based Bridge Deformation Monitoring , 2017, Front. Built Environ..

[82]  Michael A. Sutton,et al.  The effect of out-of-plane motion on 2D and 3D digital image correlation measurements , 2008 .

[83]  Maria Q. Feng,et al.  Cost‐effective vision‐based system for monitoring dynamic response of civil engineering structures , 2010 .

[84]  Jinping Ou,et al.  Free vibration monitoring experiment of a stayed-cable model based on stereovision , 2015 .

[85]  Jie Guo,et al.  Dynamic displacement measurement of large-scale structures based on the Lucas–Kanade template tracking algorithm , 2016 .

[86]  P. S. Heyns,et al.  Accuracy evaluation of sub-pixel structural vibration measurements through optical flow analysis of a video sequence , 2017 .

[87]  Byung-Wan Jo,et al.  An Efficient Vision-Based Three-Dimensional Motion Measurement System for Civil Infra-Structures , 2014 .

[88]  Thomas B. Moeslund,et al.  A Survey of Computer Vision-Based Human Motion Capture , 2001, Comput. Vis. Image Underst..

[89]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[90]  Frédo Durand,et al.  Phase-based video motion processing , 2013, ACM Trans. Graph..

[91]  José Marecos,et al.  Field Observation of Tagus River Suspension Bridge , 1969 .

[92]  David V. Jáuregui,et al.  Noncontact Photogrammetric Measurement of Vertical Bridge Deflection , 2003 .

[93]  Rui Calçada,et al.  Non-contact measurement of the dynamic displacement of railway bridges using an advanced video-based system , 2014 .

[94]  F. Necati Catbas,et al.  Computer vision-based displacement and vibration monitoring without using physical target on structures , 2017, Bridge Design, Assessment and Monitoring.

[95]  Masanobu Shinozuka,et al.  Evaluation of Bridge Load Carrying Capacity Based on Dynamic Displacement Measurement Using Real-time Image Processing Techniques , 2006 .

[96]  Lijun Wu,et al.  Local positioning systems versus structural monitoring: a review , 2014 .

[97]  Takeo Kanade,et al.  An Iterative Image Registration Technique with an Application to Stereo Vision , 1981, IJCAI.

[98]  John H G Macdonald,et al.  Dynamic measurements of the Second Severn Crossing , 1997 .

[99]  F. Necati Catbas,et al.  Completely contactless structural health monitoring of real‐life structures using cameras and computer vision , 2017 .

[100]  Kenneth R. White,et al.  Close-range photogrammetry applications in bridge measurement: Literature review , 2008 .

[101]  Maria Q. Feng,et al.  Nontarget Vision Sensor for Remote Measurement of Bridge Dynamic Response , 2015 .

[102]  Jung-Hoon Kim,et al.  3D displacement measurement model for health monitoring of structures using a motion capture system , 2015 .

[103]  Werner Lienhart,et al.  Development and evaluation of a long range image-based monitoring system for civil engineering structures , 2015, Smart Structures.