Computer Vision Techniques in Construction, Operation and Maintenance Phases of Civil Assets: A Critical Review

Throughout the life cycle of civil assets, construction, operation and maintenance phases require monitoring to assure reasonable decision makings. Current methods always involve speciallyassigned personnel conducting on-site inspections, which are work-intensive, time-consuming and errorprone. Computer vision, as a powerful alternative to manual inspection, has been extensively studied during the past decades. On the basis of existing summary papers, this paper reviews a wide range of literatures, including journal articles, conference proceedings and other resources. Current applications of computer vision during construction, operation and maintenance stages of civil structures are concluded, with a special focus on operation and maintenance phase. This review aims to provide a comprehensive insight about the utilization of computer vision in civil engineering and an inspiring guidance for future research.

[1]  KhalooAli,et al.  Robust normal estimation and region growing segmentation of infrastructure 3D point cloud models , 2017 .

[2]  Shuai Li,et al.  Vision-Based Excavator Detection and Tracking Using Hybrid Kinematic Shapes and Key Nodes , 2017, J. Comput. Civ. Eng..

[3]  Sehwan Kim,et al.  Vision-Based Natural Frequency Identification Using Laser Speckle Imaging and Parallel Computing , 2018, Comput. Aided Civ. Infrastructure Eng..

[4]  Khurram Kamal,et al.  Wood defects classification using laws texture energy measures and supervised learning approach , 2017, Adv. Eng. Informatics.

[5]  Peter E.D. Love,et al.  Falls from heights: A computer vision-based approach for safety harness detection , 2018, Automation in Construction.

[6]  Marc O. Eberhard,et al.  Bridge Column Maximum Drift Estimation via Computer Vision , 2016 .

[7]  Seokho Chi,et al.  Interaction analysis for vision-based activity identification of earthmoving excavators and dump trucks , 2018 .

[8]  SangUk Han,et al.  A vision-based motion capture and recognition framework for behavior-based safety management , 2013 .

[9]  Hua Yang,et al.  Vibration Sensing of a Bridge Model Using a Multithread Active Vision System , 2017, IEEE/ASME Transactions on Mechatronics.

[10]  Nikolaos Doulamis,et al.  Deep Convolutional Neural Networks for efficient vision based tunnel inspection , 2015, 2015 IEEE International Conference on Intelligent Computer Communication and Processing (ICCP).

[11]  Christoph Mertz,et al.  Vision for road inspection , 2014, IEEE Winter Conference on Applications of Computer Vision.

[12]  Tarek Zayed,et al.  Computer Vision-Based Model for Moisture Marks Detection and Recognition in Subway Networks , 2018, J. Comput. Civ. Eng..

[13]  Bo Xiao,et al.  Two-Dimensional Visual Tracking in Construction Scenarios: A Comparative Study , 2018, J. Comput. Civ. Eng..

[14]  Paul W. Fieguth,et al.  A review on computer vision based defect detection and condition assessment of concrete and asphalt civil infrastructure , 2015, Adv. Eng. Informatics.

[15]  Yimin D. Zhang,et al.  Road Crack Detection Using Deep Convolutional Neural Network and Adaptive Thresholding , 2019, 2019 IEEE Intelligent Vehicles Symposium (IV).

[16]  Hao Wu,et al.  An intelligent vision-based approach for helmet identification for work safety , 2018, Comput. Ind..

[17]  Alex Albert,et al.  Automating and scaling personalized safety training using eye-tracking data , 2018, Automation in Construction.

[18]  Xiaochun Luo,et al.  Automatic Pixel‐Level Crack Detection and Measurement Using Fully Convolutional Network , 2018, Comput. Aided Civ. Infrastructure Eng..

[19]  Muhammad Haroon Yousaf,et al.  Visual analysis of asphalt pavement for detection and localization of potholes , 2018, Adv. Eng. Informatics.

[20]  Yamin Li,et al.  Accurate Structural Dynamic Response Monitoring of Multiple Structures using One CCD Camera and a Novel Targets Configuration , 2017 .

[21]  Ali Khaloo,et al.  Robust normal estimation and region growing segmentation of infrastructure 3D point cloud models , 2017, Adv. Eng. Informatics.

[22]  Reginald DesRoches,et al.  Machine Vision-Enhanced Postearthquake Inspection , 2011, J. Comput. Civ. Eng..

[23]  Ioannis Brilakis,et al.  Comparative study of vision tracking methods for tracking of construction site resources , 2011 .

[24]  Carl J. Debono,et al.  Vision-based change detection for inspection of tunnel liners , 2018, Automation in Construction.

[25]  Wei-Min Shen,et al.  Progressive image stitching algorithm for vision based automated inspection , 2016, 2016 International Conference on Machine Learning and Cybernetics (ICMLC).

[26]  J. Nathan Kutz,et al.  Data-driven vision-based inspection for reinforced concrete beams and slabs: Quantitative damage and load estimation , 2018, Automation in Construction.

[27]  Juan Carlos Niebles,et al.  Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers , 2013, Adv. Eng. Informatics.

[28]  Zhongke Shi,et al.  Vision-Based Tower Crane Tracking for Understanding Construction Activity , 2014, J. Comput. Civ. Eng..

[29]  Maria Q. Feng,et al.  Edge‐Enhanced Matching for Gradient‐Based Computer Vision Displacement Measurement , 2018, Comput. Aided Civ. Infrastructure Eng..

[30]  Thomas J. Armstrong,et al.  Motion Data-Driven Biomechanical Analysis during Construction Tasks on Sites , 2015 .

[31]  Patricio A. Vela,et al.  Construction performance monitoring via still images, time-lapse photos, and video streams: Now, tomorrow, and the future , 2015, Adv. Eng. Informatics.

[32]  Bahman Jafari,et al.  Tracking Structural Deformations via Automated Sample-Based Point Cloud Analysis , 2017 .

[33]  Sandra Cancino Suárez,et al.  Material deformation estimation with Computer Vision methods , 2015 .

[34]  Xiaoling Zhang,et al.  Construction waste recycling robot for nails and screws: Computer vision technology and neural network approach , 2019, Automation in Construction.

[35]  Carlos Balaguer,et al.  Tunnel structural inspection and assessment using an autonomous robotic system , 2018 .

[36]  Peter E.D. Love,et al.  Convolutional neural networks: Computer vision-based workforce activity assessment in construction , 2018, Automation in Construction.

[37]  Amin Hammad,et al.  Framework for Location Data Fusion and Pose Estimation of Excavators Using Stereo Vision , 2018, J. Comput. Civ. Eng..

[38]  F. Necati Catbas,et al.  Completely contactless structural health monitoring of real‐life structures using cameras and computer vision , 2017 .

[39]  Mohammad R. Jahanshahi,et al.  An innovative methodology for detection and quantification of cracks through incorporation of depth perception , 2011, Machine Vision and Applications.

[40]  Kinam Kim,et al.  Vision-Based Object-Centric Safety Assessment Using Fuzzy Inference: Monitoring Struck-By Accidents with Moving Objects , 2016, J. Comput. Civ. Eng..

[41]  Francesco Benedetto,et al.  A real-time automatic pavement crack and pothole recognition system for mobile Android-based devices , 2017, Adv. Eng. Informatics.

[42]  Katsuto Yamaki,et al.  Road deformation detection based sensor fusion , 2017, 2017 9th International Conference on Information Technology and Electrical Engineering (ICITEE).

[43]  Myra Lydon,et al.  Development and Field Testing of a Time-Synchronized System for Multi-Point Displacement Calculation Using Low-Cost Wireless Vision-Based Sensors , 2018, IEEE Sensors Journal.

[44]  Muhammad Haroon Yousaf,et al.  Computer vision based detection and localization of potholes in asphalt pavement images , 2016, 2016 IEEE Canadian Conference on Electrical and Computer Engineering (CCECE).

[45]  Angelos Amditis,et al.  Crack Identification Via User Feedback, Convolutional Neural Networks and Laser Scanners for Tunnel Infrastructures , 2016, VISIGRAPP.

[46]  Jack Chin Pang Cheng,et al.  Automated detection of sewer pipe defects in closed-circuit television images using deep learning techniques , 2018, Automation in Construction.

[47]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[48]  Paul Fieguth,et al.  Computer Vision Techniques for Automatic Structural Assessment of Underground Pipes , 2003 .

[49]  Sidney Nascimento Givigi,et al.  Automatic Crack Detection and Measurement Based on Image Analysis , 2016, IEEE Transactions on Instrumentation and Measurement.

[50]  Chul Min Yeum,et al.  Vision‐Based Automated Crack Detection for Bridge Inspection , 2015, Comput. Aided Civ. Infrastructure Eng..

[51]  Takashi Matsumoto,et al.  Development of an Automatic Detector of Cracks in Concrete Using Machine Learning , 2017 .

[52]  I. Iervolino,et al.  Computer Aided Civil and Infrastructure Engineering , 2009 .

[53]  Sami F. Masri,et al.  Adaptive vision-based crack detection using 3D scene reconstruction for condition assessment of structures , 2012 .

[54]  Jian Li,et al.  Vision‐Based Fatigue Crack Detection of Steel Structures Using Video Feature Tracking , 2018, Comput. Aided Civ. Infrastructure Eng..

[55]  Yang Liu,et al.  Automated Pixel‐Level Pavement Crack Detection on 3D Asphalt Surfaces Using a Deep‐Learning Network , 2017, Comput. Aided Civ. Infrastructure Eng..

[56]  Ioannis Brilakis,et al.  Visual retrieval of concrete crack properties for automated post-earthquake structural safety evaluation , 2011 .

[57]  Maria Q. Feng,et al.  Vision-Based Displacement Sensor for Monitoring Dynamic Response Using Robust Object Search Algorithm , 2013 .

[58]  Yang Yang,et al.  Real time measurement of the dynamic displacement field of a large-scale arch-truss bridge by remote sensing technology , 2016, 2016 IEEE SENSORS.

[59]  Robert J. Thomas,et al.  Fatigue Crack Detection Using Unmanned Aerial Systems in Fracture Critical Inspection of Steel Bridges , 2018, Journal of Bridge Engineering.

[60]  Oral Büyüköztürk,et al.  Autonomous Structural Visual Inspection Using Region‐Based Deep Learning for Detecting Multiple Damage Types , 2018, Comput. Aided Civ. Infrastructure Eng..

[61]  Wei Zhang,et al.  Unified Vision‐Based Methodology for Simultaneous Concrete Defect Detection and Geolocalization , 2018, Comput. Aided Civ. Infrastructure Eng..

[62]  Kaige Zhang,et al.  Unified Approach to Pavement Crack and Sealed Crack Detection Using Preclassification Based on Transfer Learning , 2018, J. Comput. Civ. Eng..

[63]  Hae-Bum Yun,et al.  Automatic Pavement Object Detection Using Superpixel Segmentation Combined With Conditional Random Field , 2018, IEEE Transactions on Intelligent Transportation Systems.

[64]  Hou Qizhen,et al.  On image edge detection method , 2011, 2011 International Conference on Electrical and Control Engineering.

[65]  Jitesh J. Dhule,et al.  Edge Detection Technique Used for Identification of Cracks on Vertical Walls of The Building , 2015, 2015 International Conference on Computing and Network Communications (CoCoNet).

[66]  Reginald DesRoches,et al.  Rapid entropy-based detection and properties measurement of concrete spalling with machine vision for post-earthquake safety assessments , 2012, Adv. Eng. Informatics.

[67]  Heng Li,et al.  Computer vision aided inspection on falling prevention measures for steeplejacks in an aerial environment , 2018, Automation in Construction.

[68]  Young-Jin Cha,et al.  Vision-based detection of loosened bolts using the Hough transform and support vector machines , 2016 .

[69]  Xiaochun Luo,et al.  Recognizing Diverse Construction Activities in Site Images via Relevance Networks of Construction-Related Objects Detected by Convolutional Neural Networks , 2018, J. Comput. Civ. Eng..