Remote proximity monitoring between mobile construction resources using camera-mounted UAVs

Abstract Struck-by accidents have resulted in a significant number of fatal and nonfatal injuries in the construction industry. As a proactive safety measure against struck-by hazards, the authors present an Unmanned Aerial Vehicle (UAV)-assisted visual monitoring method that can automatically measure proximities among construction entities. To attain this end, this research conducts two research thrusts: (i) object localization using a deep neural network, YOLO-V3; and (ii) development of an image rectification method that allows for the measurement of actual distance from a 2D image collected from a UAV. Tests on real-site aerial videos show the promising accuracy of the proposed method; the mean absolute distance errors for estimated proximity were less than 0.9 m and the mean absolute percentage errors were around 4%. The proposed method enables the advanced detection of struck-by hazards around workers, which in turn can make timely intervention possible. This proactive intervention can ultimately promote a safer working environment for construction workers.

[1]  Kinam Kim,et al.  Vision-Based Object-Centric Safety Assessment Using Fuzzy Inference: Monitoring Struck-By Accidents with Moving Objects , 2016, J. Comput. Civ. Eng..

[2]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[3]  Ioannis K. Brilakis,et al.  Automated vision tracking of project related entities , 2011, Adv. Eng. Informatics.

[4]  Kinam Kim,et al.  Image-based construction hazard avoidance system using augmented reality in wearable device , 2017 .

[5]  Kazuya Yoshida,et al.  Collaborative mapping of an earthquake‐damaged building via ground and aerial robots , 2012, J. Field Robotics.

[6]  Jitendra Malik,et al.  Region-Based Convolutional Networks for Accurate Object Detection and Segmentation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Yong K. Cho,et al.  Improving dynamic proximity sensing and processing for smart work-zone safety , 2017 .

[8]  Jimmie Hinze,et al.  Autonomous pro-active real-time construction worker and equipment operator proximity safety alert system , 2010 .

[9]  Ioannis Brilakis,et al.  Continuous localization of construction workers via integration of detection and tracking , 2016 .

[10]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  Horst Bischof,et al.  Augmented Reality for Construction Site Monitoring and Documentation , 2014, Proceedings of the IEEE.

[12]  Ioannis Brilakis,et al.  Construction worker detection in video frames for initializing vision trackers , 2012 .

[13]  Norman Kerle,et al.  Urban structural damage assessment with oblique UAV imagery, object-based image analysis and semantic reasoning , 2014 .

[14]  SangHyun Lee,et al.  Computer vision techniques for construction safety and health monitoring , 2015, Adv. Eng. Informatics.

[15]  Eric Marks,et al.  Proximity Sensing and Warning Technology for Heavy Construction Equipment Operation , 2012 .

[16]  Yi-Ping Chen,et al.  Image-based 3D scene reconstruction and exploration in augmented reality , 2013 .

[17]  Anu Pradhan,et al.  Automated Detection of Damaged Areas after Hurricane Sandy using Aerial Color Images , 2014 .

[18]  Ioannis Brilakis,et al.  Comparative study of vision tracking methods for tracking of construction site resources , 2011 .

[19]  Xiaochun Luo,et al.  Detecting non-hardhat-use by a deep learning method from far-field surveillance videos , 2018 .

[20]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2015, IEEE Trans. Pattern Anal. Mach. Intell..

[21]  Eric Marks,et al.  Performance Test of Wireless Technologies for Personnel and Equipment Proximity Sensing in Work Zones , 2015 .

[22]  Jochen Teizer.  Wearable, wireless identification sensing platform: Self-Monitoring Alert and Reporting Technology for Hazard Avoidance and Training (SmartHat) , 2015, J. Inf. Technol. Constr..

[23]  Norman Kerle,et al.  UAV-based urban structural damage assessment using object-based image analysis and semantic reasoning , 2014 .

[24]  Seokho Chi,et al.  Image-Based Safety Assessment: Automated Spatial Safety Risk Identification of Earthmoving and Surface Mining Activities , 2012 .

[25]  Chunsun Zhang,et al.  An Unmanned Aerial Vehicle‐Based Imaging System for 3D Measurement of Unpaved Road Surface Distresses 1 , 2012, Comput. Aided Civ. Infrastructure Eng..

[26]  Burcin Becerik-Gerber,et al.  A Data Quality-Driven Framework for Asset Condition Assessment Using LiDAR and Image Data , 2015 .

[27]  Patricio A. Vela,et al.  Personnel tracking on construction sites using video cameras , 2009, Adv. Eng. Informatics.

[28]  Vineet R. Kamat,et al.  Feasibility of a Drone-Based On-Site Proximity Detection in an Outdoor Construction Site , 2017 .

[29]  Xiaowei Luo,et al.  Transfer learning and deep convolutional neural networks for safety guardrail detection in 2D images , 2018 .

[30]  Youngjib Ham,et al.  Analyzing context and productivity of tunnel earthmoving processes using imaging and simulation , 2018, Automation in Construction.

[31]  Zhongke Shi,et al.  Tracking multiple workers on construction sites using video cameras , 2010, Adv. Eng. Informatics.

[32]  Mani Golparvar-Fard,et al.  Automated 2D detection of construction equipment and workers from site video streams using histograms of oriented gradients and colors , 2013 .

[33]  Zhi Chen,et al.  Integrated detection and tracking of workforce and equipment from construction jobsite videos , 2017 .

[34]  Richard J. Dobson,et al.  Developing an unpaved road assessment system for practical deployment with high-resolution optical data collection using a helicopter UAV , 2013, 2013 International Conference on Unmanned Aircraft Systems (ICUAS).

[35]  O. Hellwich,et al.  Three-Dimensional Building Reconstruction Using Images Obtained by Unmanned Aerial Vehicles , 2012 .

[36]  Mani Golparvar-Fard,et al.  Visual monitoring of civil infrastructure systems via camera-equipped Unmanned Aerial Vehicles (UAVs): a review of related works , 2016 .