A Review of Machine Vision-Based Structural Health Monitoring: Methodologies and Applications

In the past two decades, a significant number of innovative sensing and monitoring systems based on the machine vision-based technology have been exploited in the field of structural health monitoring (SHM). This technology has some inherent distinctive advantages such as noncontact, nondestructive, long distance, high precision, immunity to electromagnetic interference, and large-range and multiple-target monitoring. A lot of machine vision-based structural dynamic measurement and structural state inspection methods have been proposed. Real-world applications are also carried out to measure the structural physical parameters such as the displacement, strain/stress, rotation, vibration, crack, and spalling. The purpose of this review article is devoted to presenting a summary of the basic theories and practical applications of the machine vision-based technology employed in structural monitoring as well as its systematic error sources and integration with other modern sensing techniques.

[1]  Byeong Hwa Kim,et al.  Extracting modal parameters of a cable on shaky motion pictures , 2014 .

[2]  J. G. Chase,et al.  Seismic structural displacement measurement using a line-scan camera: camera-pattern calibration and experimental validation , 2011 .

[3]  H. Haddadi,et al.  Use of rigid-body motion for the investigation and estimation of the measurement errors related to digital image correlation technique , 2008 .

[4]  Chih-Chen Chang,et al.  Three-Dimensional Structural Translation and Rotation Measurement Using Monocular Videogrammetry , 2010 .

[5]  Maurice Whelan,et al.  Calibration and evaluation of optical systems for full-field strain measurement , 2007 .

[6]  Y. Wang,et al.  Error estimation in measuring strain fields with DIC on planar sheet metal specimens with a non-perpendicular camera alignment , 2011 .

[7]  Piotr Kohut,et al.  Application of modal analysis supported by 3D vision-based measurements , 2009 .

[8]  Carlo Atzeni,et al.  Remote sensing of building structural displacements using a microwave interferometer with imaging capability , 2004 .

[9]  Hiroshi Hattori,et al.  Image-Based Monitoring of Open Gears of Movable Bridges for Condition Assessment and Maintenance Decision Making , 2015 .

[10]  Hisao Kikuta,et al.  Lens distortion correction for digital image correlation by measuring rigid body displacement , 2006 .

[11]  Hyun Myung,et al.  Vision-based remote 6-DOF structural displacement monitoring system using a unique marker , 2014 .

[12]  Maria Q. Feng,et al.  Vision-Based Displacement Sensor for Monitoring Dynamic Response Using Robust Object Search Algorithm , 2013 .

[13]  Mohammed Taleb Obaidat,et al.  Computer Vision-Based Technique to Measure Displacement in Selected Soil Tests , 1998 .

[14]  Stefano Pignatti,et al.  A new joint application of non-invasive remote sensing techniques for structural health monitoring , 2012 .

[15]  Stefan Winter,et al.  Use of digital image processing in the monitoring of deformations in building structures , 2015 .

[16]  M. Sutton,et al.  Systematic errors in digital image correlation due to undermatched subset shape functions , 2002 .

[17]  M. Grédiac,et al.  Assessment of Digital Image Correlation Measurement Errors: Methodology and Results , 2009 .

[18]  Kenneth R. White,et al.  Close-range photogrammetry applications in bridge measurement: Literature review , 2008 .

[19]  John Tyson,et al.  Full-field dynamic displacement and strain measurement—Specific examples using advanced 3D image correlation photogrammetry: Part II , 2003 .

[20]  Mahmoud R. Halfawy,et al.  Integrated Vision-Based System for Automated Defect Detection in Sewer Closed Circuit Television Inspection Videos , 2015, J. Comput. Civ. Eng..

[21]  C. Fraser.  Some Thoughts on the Emergence of Digital Close Range Photogrammetry , 1998 .

[22]  Falko Kuester,et al.  Monitoring global earthquake-induced demands using vision-based sensors , 2004, IEEE Transactions on Instrumentation and Measurement.

[23]  Sami F. Masri,et al.  A vision-based approach for the direct measurement of displacements in vibrating systems , 2003 .

[24]  Tadeusz Uhl,et al.  Monitoring of a civil structure’s state based on noncontact measurements , 2013 .

[25]  Chun Cheng,et al.  A preliminary study on the response of steel structures using surveillance camera image with vision-based method during the Great East Japan Earthquake , 2015 .

[26]  Jack Leifer,et al.  Three-Dimensional Acceleration Measurement Using Videogrammetry Tracking Data , 2011 .

[27]  Peter Gosling,et al.  Videogrammetric monitoring of as‐built membrane roof structures , 2008 .

[28]  Chih-Chen Chang,et al.  Flexible Videogrammetric Technique for Three-Dimensional Structural Vibration Measurement , 2007 .

[29]  Billie F. Spencer,et al.  Concrete Crack Assessment Using Digital Image Processing and 3D Scene Reconstruction , 2016, J. Comput. Civ. Eng..

[30]  Yong Xia,et al.  Measurement of rivulet movement on inclined cables during rain–wind induced vibration , 2015 .

[31]  Rajalingappaa Shanmugamani,et al.  Detection and classification of surface defects of gun barrels using computer vision and machine learning , 2015 .

[32]  Gaurav S. Sukhatme,et al.  Multi-image stitching and scene reconstruction for evaluating defect evolution in structures , 2011 .

[33]  Anand Asundi,et al.  A four-camera videogrammetric system for 3-D motion measurement of deformable object , 2012 .

[34]  Thierry Viéville,et al.  Using Specific Displacements to Analyze Motion without Calibration , 1999, International Journal of Computer Vision.

[35]  Qi Li,et al.  Restoration of TDI camera images with motion distortion and blur , 2010 .

[36]  Jun Ueda,et al.  Dynamics-based motion de-blurring for a PZT-driven, compliant camera orientation mechanism , 2015, Int. J. Robotics Res..

[37]  Jong-Jae Lee,et al.  A vision-based system for remote sensing of bridge displacement , 2006 .

[38]  Masanobu Shinozuka,et al.  Evaluation of Bridge Load Carrying Capacity Based on Dynamic Displacement Measurement Using Real-time Image Processing Techniques , 2006 .

[39]  X. W. Ye,et al.  A vision-based system for dynamic displacement measurement of long-span bridges : algorithm and verification , 2013 .

[40]  Rui Calçada,et al.  Non-contact measurement of the dynamic displacement of railway bridges using an advanced video-based system , 2014 .

[41]  David V. Jáuregui,et al.  Noncontact Photogrammetric Measurement of Vertical Bridge Deflection , 2003 .

[42]  Jin-Hee Ahn,et al.  Structural dynamic displacement vision system using digital image processing , 2011 .

[43]  Billie F. Spencer,et al.  Automated assessment of cracks on concrete surfaces using adaptive digital image processing , 2014 .

[44]  Hyo Seon Park,et al.  Vision-based system identification technique for building structures using a motion capture system , 2015 .

[45]  X. W. Ye,et al.  Image-based structural dynamic displacement measurement using different multi-object tracking algorithms , 2016 .

[46]  José Machado,et al.  Yarn features extraction using image processing and computer vision – A study with cotton and polyester yarns , 2015 .

[47]  Jong-Han Lee,et al.  An advanced vision-based system for real-time displacement measurement of high-rise buildings , 2012 .

[48]  Jianping Wu,et al.  Horizontal Roadway Curvature Computation Algorithm Using Vision Technology , 2010, Comput. Aided Civ. Infrastructure Eng..

[49]  X. W. Ye,et al.  Structural Health Monitoring of Civil Infrastructure Using Optical Fiber Sensing Technology : A Comprehensive Review , 2015 .

[50]  Young-Chul Choi,et al.  MULTI-POINT MEASUREMENT OF STRUCTURAL VIBRATION USING PATTERN RECOGNITION FROM CAMERA IMAGE , 2010 .

[51]  Abdenour Nazef,et al.  Improvement of Crack-Detection Accuracy Using a Novel Crack Defragmentation Technique in Image-Based Road Assessment , 2016 .

[52]  Tadeusz Uhl,et al.  Vision‐based algorithms for damage detection and localization in structural health monitoring , 2016 .

[53]  Jr Eyre,et al.  Vision metrology and three dimensional visualization in structural testing and monitoring , 1999 .

[54]  Demeke B. Ashebo,et al.  Vertical Displacement Measurements for Bridges Using Optical Fiber Sensors and CCD Cameras — A Preliminary Study , 2009 .

[55]  Jeong-Tae Kim,et al.  Vision-based technique for bolt-loosening detection in wind turbine tower , 2015 .

[56]  Stephen Boyd,et al.  Speckle pattern quality assessment for digital image correlation , 2013 .

[57]  X. W. Ye,et al.  Statistical analysis and probabilistic modeling of WIM monitoring data of an instrumented arch bridge , 2016 .

[58]  Sven Bossuyt,et al.  Quality assessment of speckle patterns for digital image correlation , 2006 .

[59]  Mikael Sjödahl,et al.  A stereoscopic digital speckle photography system for 3-D displacement field measurements , 1999 .

[60]  X. W. Ye,et al.  Monitoring-Based Fatigue Reliability Assessment of Steel Bridges : Analytical Model and Application , 2010 .

[61]  M. A. Sutton,et al.  Systematic errors in digital image correlation caused by intensity interpolation , 2000 .

[62]  Zhu Dachang,et al.  Image recognition technology in rotating machinery fault diagnosis based on artificial immune , 2010 .

[63]  Eduardo Júlio,et al.  Automatic concrete health monitoring: assessment and monitoring of concrete surfaces , 2014 .

[64]  Erwin Hack,et al.  Camera-based monitoring of the rigid-body displacement of a mandrel in superconducting cable production , 2005 .

[65]  Ting-Hua Yi,et al.  Multi-point displacement monitoring of bridges using a vision-based approach , 2015 .

[66]  Daniel Leonardo B.R. Jurjo,et al.  Analysis of the structural behavior of a membrane using digital image processing , 2015 .

[67]  Gwolong Lai,et al.  Application of digital photogrammetry techniques in identifying the mode shape ratios of stay cables with multiple camcorders , 2015 .

[68]  Reginald DesRoches,et al.  Machine Vision-Enhanced Postearthquake Inspection , 2011, J. Comput. Civ. Eng..

[69]  Wim De Waele,et al.  On the use of digital image correlation for slip measurement during coupon scale fretting fatigue experiments , 2014 .

[70]  Carolina Moilanen,et al.  Image-based stress and strain measurement of wood in the split-Hopkinson pressure bar , 2015 .

[71]  Jie Guo,et al.  Dynamic displacement measurement of large-scale structures based on the Lucas–Kanade template tracking algorithm , 2016 .

[72]  Maria Q. Feng,et al.  A Vision-Based Sensor for Noncontact Structural Displacement Measurement , 2015, Sensors.

[73]  Raffaele Zinno,et al.  A new low-cost displacements monitoring system based on Kinect sensor , 2015 .

[74]  R Zaurin,et al.  Integration of computer imaging and sensor data for structural health monitoring of bridges , 2009 .

[75]  Chen Sun,et al.  Adaptive subset offset for systematic error reduction in incremental digital image correlation , 2014 .

[76]  Hiroshi Murase,et al.  Recognition of camera-captured low-quality characters using motion blur information , 2008, Pattern Recognit..

[77]  Chung Bang Yun,et al.  Development and application of a vision-based displacement measurement system for structural health monitoring of civil structures , 2006 .

[78]  R.N.F. Carmo,et al.  Assessing steel strains on reinforced concrete members from surface cracking patterns , 2015 .

[79]  Mustafa Gul,et al.  Sensor Networks, Computer Imaging, and Unit Influence Lines for Structural Health Monitoring: Case Study for Bridge Load Rating , 2012 .

[80]  Young-Soo Park,et al.  An efficient image-based damage detection for cable surface in cable-stayed bridges , 2013 .

[81]  Michiyasu Noda,et al.  Experimental validation of non-contacting measurement method using LED-optical displacement sensors for vibration stress of small-bore piping , 2015 .

[82]  Elsa Caetano,et al.  A vision system for vibration monitoring of civil engineering structures , 2011 .

[83]  Luke Bisby,et al.  New Parameters to Describe High-Temperature Deformation of Prestressing Steel Determined Using Digital Image Correlation , 2012 .

[84]  Markus König,et al.  Achievements and Challenges in Machine Vision-Based Inspection of Large Concrete Structures , 2014 .

[85]  Kim Branner,et al.  Strain and Displacement Controls by Fibre Bragg Grating and Digital Image Correlation , 2014 .

[86]  Keigo Hirakawa,et al.  Blind Deblurring and Denoising of Images Corrupted by Unidirectional Object Motion Blur and Sensor Noise , 2016, IEEE Transactions on Image Processing.

[87]  M. J. McGinnis,et al.  Application of three-dimensional digital image correlation to the core-drilling method , 2005 .

[88]  Hyun Myung,et al.  Vision-based displacement measurement method for high-rise building structures using partitioning approach , 2010 .

[89]  Chih-Chen Chang,et al.  Nontarget Stereo Vision Technique for Spatiotemporal Response Measurement of Line-Like Structures , 2008 .

[90]  Gongkang Fu,et al.  AN OPTICAL APPROACH TO STRUCTURAL DISPLACEMENT MEASUREMENT AND ITS APPLICATION , 2002 .

[91]  Shaopeng Ma,et al.  The systematic error in digital image correlation induced by self-heating of a digital camera , 2012 .

[92]  Hyun Myung,et al.  Pose-graph optimized displacement estimation for structural displacement monitoring , 2014 .

[93]  Emanuele Zappa,et al.  Vision-based estimation of vertical dynamic loading induced by jumping and bobbing crowds on civil structures , 2012 .

[94]  Sun Yaofeng,et al.  Study of optimal subset size in digital image correlation of speckle pattern images , 2007 .

[95]  Lijun Wu,et al.  Dynamic testing of a laboratory model via vision-based sensing , 2014 .

[96]  Antonio Baldi,et al.  Experimental Analysis of the Errors due to Polynomial Interpolation in Digital Image Correlation , 2015 .

[97]  Hyo Seon Park,et al.  A New Position Measurement System Using a Motion-Capture Camera for Wind Tunnel Tests , 2013, Sensors.

[98]  Perng-Fei Luo,et al.  Application of computer vision and laser interferometer to the inspection of line scale , 2004 .

[99]  Paul W. Fieguth,et al.  A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure , 2015, Adv. Eng. Informatics.

[100]  Jung-Hoon Kim,et al.  3D displacement measurement model for health monitoring of structures using a motion capture system , 2015 .

[101]  John Lambros,et al.  High resolution digital image correlation measurements of strain accumulation in fatigue crack growth , 2013 .

[102]  Ting-Hua Yi,et al.  Vision-based structural displacement measurement: System performance evaluation and influence factor analysis , 2016 .

[103]  Chul Min Yeum,et al.  Vision‐Based Automated Crack Detection for Bridge Inspection , 2015, Comput. Aided Civ. Infrastructure Eng..

[104]  A. Cuitiño,et al.  Full-field measurements of heterogeneous deformation patterns on polymeric foams using digital image correlation , 2002 .

[105]  Ashutosh Bagchi,et al.  Automated condition assessment of concrete bridges with digital imaging , 2014 .

[106]  X. W. Ye,et al.  Force monitoring of steel cables using vision-based sensing technology: methodology and experimental verification , 2016 .

[107]  Jong-Jae Lee,et al.  Real-Time Displacement Measurement of a Flexible Bridge Using Digital Image Processing Techniques , 2006 .

[108]  Piotr Olaszek,et al.  Investigation of the dynamic characteristic of bridge structures using a computer vision method , 1999 .

[109]  Lei Lei,et al.  Study on Three-Dimensional Displacement Measurement Method of EAST Cold Magnet Based on Computer Vision , 2014 .

[110]  Yi-Qing Ni,et al.  Modeling of Stress Spectrum Using Long-Term Monitoring Data and Finite Mixture Distributions , 2012 .

[111]  John Tyson,et al.  Pull-field dynamic displacement and strain measurement using advanced 3D image correlation photogrammetry: Part 1 , 2003 .

[112]  Michael E. Becker.  Motion-blur evaluation: A comparison of approaches , 2008 .

[113]  Vincenzo Caglioti,et al.  Recovering ball motion from a single motion-blurred image , 2009, Comput. Vis. Image Underst..

[114]  Andrew W. Smyth,et al.  Monocular Computer Vision Method for the Experimental Study of Three-Dimensional Rocking Motion , 2016 .

[115]  S. Patsias,et al.  Damage Detection Using Optical Measurements and Wavelets , 2002 .

[116]  Olivier Dalverny,et al.  Study of image characteristics on digital image correlation error assessment , 2010 .

[117]  I. Brilakis,et al.  Multistep Explicit Stereo Camera Calibration Approach to Improve Euclidean Accuracy of Large-Scale 3D Reconstruction , 2016 .

[118]  Tonghai Wu,et al.  Motion-Blurred Particle Image Restoration for On-Line Wear Monitoring , 2015, Sensors.

[119]  David J. Whitehouse,et al.  Phase-shifted image matching algorithm for displacement measurement , 1996 .

[120]  Hisao Kikuta,et al.  Bridge deflection measurement using digital image correlation , 2007 .

[121]  Jorge Batista,et al.  Long Deck Suspension Bridge Monitoring: The Vision System Calibration Problem , 2012 .

[122]  Guobao Wang,et al.  Measurement of sinusoidal vibration from motion blurred images , 2007, Pattern Recognit. Lett..

[123]  Jorge Batista,et al.  Calibration methodology of a vision system for measuring the displacements of long‐deck suspension bridges , 2012 .

[124]  Dongyeon Lee,et al.  Experimental Study of Dynamic Crack Growth in Unidirectional Graphite/Epoxy Composites using Digital Image Correlation Method and High-speed Photography , 2009 .

[125]  Chang-Wei Huang,et al.  A simple image‐based strain measurement method for measuring the strain fields in an RC‐wall experiment , 2012 .

[126]  Dimitri Debruyne,et al.  Assessment of measuring errors in DIC using deformation fields generated by plastic FEA , 2009 .

[127]  Emanuele Zappa,et al.  Vibration Monitoring of Multiple Bridge Points by Means of a Unique Vision-Based Measuring System , 2014 .

[128]  Dimitri Debruyne,et al.  Study of systematic errors in strain fields obtained via DIC using heterogeneous deformation generated by plastic FEA , 2010 .

[129]  Maria Q. Feng,et al.  Cost‐effective vision‐based system for monitoring dynamic response of civil engineering structures , 2010 .

[130]  B. Pan,et al.  The errors in digital image correlation due to overmatched shape functions , 2015 .

[131]  Jan Winkler,et al.  Measurement of Local Deformations in Steel Monostrands Using Digital Image Correlation , 2014 .

[132]  Chih-Chen Chang,et al.  Nontarget Image-Based Technique for Small Cable Vibration Measurement , 2008 .

[133]  Daniel Leonardo B.R. Jurjo,et al.  Experimental methodology for the dynamic analysis of slender structures based on digital image processing techniques , 2010 .

[134]  Devin K. Harris,et al.  Combined Imaging Technologies for Concrete Bridge Deck Condition Assessment , 2015 .

[135]  J. Ye,et al.  Structural damage detection using digital video imaging technique and wavelet transformation , 2005 .

[136]  Wen-Ko Hsu,et al.  Smart monitoring system with multi-criteria decision using a feature based computer vision technique , 2015 .

[137]  Yi-Qing Ni,et al.  Statistical analysis of stress spectra for fatigue life assessment of steel bridges with structural health monitoring data , 2012 .

[138]  Takahide Sakagami.  Remote nondestructive evaluation technique using infrared thermography for fatigue cracks in steel bridges , 2015 .

[139]  Jianwen Liang,et al.  Digital image processing for non-linear system identification , 2004 .

[140]  X. W. Ye,et al.  Safety Monitoring of Railway Tunnel Construction Using FBG Sensing Technology , 2013 .

[141]  Lijun Wu,et al.  Local positioning systems versus structural monitoring: a review , 2014 .

[142]  Jong-Jae Lee,et al.  A Vision-Based Dynamic Rotational Angle Measurement System for Large Civil Structures , 2012, Sensors.

[143]  Young-Soo Park,et al.  A Synchronized Multipoint Vision-Based System for Displacement Measurement of Civil Infrastructures , 2012, TheScientificWorldJournal.

[144]  Hyun Myung,et al.  A paired visual servoing system for 6-DOF displacement measurement of structures , 2011 .

[145]  Chun-Hui Wang,et al.  Accurate and robust ROI localization in a camshift tracking application , 2014, Multimedia Tools and Applications.