Falls from heights: A computer vision-based approach for safety harness detection

Abstract Falls from heights (FFH) are major contributors of injuries and deaths in construction. Yet, despite workers being made aware of the dangers associated with not wearing a safety harness, many forget or purposefully do not wear them when working at heights. To address this problem, this paper develops an automated computer vision-based method that uses two convolutional neural network (CNN) models to determine if workers are wearing their harness when performing tasks while working at heights. The algorithms developed are: (1) a Faster-R-CNN to detect the presence of a worker; and (2) a deep CNN model to identify the harness. A database of photographs of people working at heights was created from activities undertaken on several construction projects in Wuhan, China. The database was then used to test and train the developed networks. The precision and recall rates for the Faster R-CNN were 99% and 95%, and the CNN models 80% and 98%, respectively. The results demonstrate that the developed method can accurately detect workers not wearing their harness. Thus, the computer vision-based approach developed can be used by construction and safety managers as a mechanism to proactively identify unsafe behavior and therefore take immediate action to mitigate the likelihood of a FFH occurring.

[1]  Seunghoon Hong,et al.  Online Tracking by Learning Discriminative Saliency Map with Convolutional Neural Network , 2015, ICML.

[2]  Peter E.D. Love,et al.  A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory , 2018 .

[3]  Sergej Muhic,et al.  SeeBridge as next generation bridge inspection: Overview, Information Delivery Manual and Model View Definition , 2018, Automation in Construction.

[4]  Kun Hu,et al.  Factors influencing the risk of falls in the construction industry: a review of the evidence , 2011 .

[5]  SangHyun Lee,et al.  Computer vision techniques for construction safety and health monitoring , 2015, Adv. Eng. Informatics.

[6]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[7]  Simon Haykin,et al.  GradientBased Learning Applied to Document Recognition , 2001 .

[8]  Kent Jacob Nielsen,et al.  A comparison of inspection practices within the construction industry between the Danish and Swedish Work Environment Authorities , 2017 .

[9]  SangUk Han,et al.  A vision-based motion capture and recognition framework for behavior-based safety management , 2013 .

[10]  Kinam Kim,et al.  Vision-Based Object-Centric Safety Assessment Using Fuzzy Inference: Monitoring Struck-By Accidents with Moving Objects , 2016, J. Comput. Civ. Eng..

[11]  F P Rivara,et al.  Prevention of falls in the construction industry: evidence for program effectiveness. , 2000, American journal of preventive medicine.

[12]  Tiago M. Fernández-Caramés,et al.  Real-time personal protective equipment monitoring system , 2012, Comput. Commun..

[13]  J. Aken.  Management Research Based on the Paradigm of the Design Sciences: The Quest for Field-Tested and Grounded Technological Rules , 2004 .

[14]  Junqing Yu,et al.  On-Device Mobile Visual Location Recognition by Using Panoramic Images and Compressed Sensing Based Visual Descriptors , 2014, PloS one.

[15]  Reem Zeibak-Shini,et al.  Information modeling of earthquake-damaged reinforced concrete structures , 2015, Adv. Eng. Informatics.

[16]  Dongping Fang,et al.  A cognitive analysis of why Chinese scaffolders do not use safety harnesses in construction , 2013 .

[17]  Feniosky Peña-Mora,et al.  Vision-Based Detection of Unsafe Actions of a Construction Worker: Case Study of Ladder Climbing , 2013, J. Comput. Civ. Eng..

[18]  Hulya Cakan,et al.  Investigation of Factors Contributing to Fatal and Nonfatal Roofer Fall Accidents , 2014 .

[19]  Junqing Yu,et al.  Real-Time Camera Pose Estimation for Wide-Area Augmented Reality Applications , 2011, IEEE Computer Graphics and Applications.

[20]  Evan A. Nadhim,et al.  Falls from Height in the Construction Industry: A Critical Review of the Scientific Literature , 2016, International journal of environmental research and public health.

[21]  Jochen Teizer,et al.  Real-time construction worker posture analysis for ergonomics training , 2012, Adv. Eng. Informatics.

[22]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  SangHyun Lee,et al.  Automated Postural Ergonomic Assessment Using a Computer Vision-Based Posture Classification , 2016 .

[24]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[25]  Rob Fergus,et al.  Visualizing and Understanding Convolutional Networks , 2013, ECCV.

[26]  Zhi Chen,et al.  Visual Tracking of Construction Jobsite Workforce and Equipment with Particle Filtering , 2016, J. Comput. Civ. Eng..

[27]  Feniosky Peña-Mora,et al.  Comparative Study of Motion Features for Similarity-Based Modeling and Classification of Unsafe Actions in Construction , 2014, J. Comput. Civ. Eng..

[28]  Timothy Bretl,et al.  Detecting and Classifying Cranes Using Camera-Equipped UAVs for Monitoring Crane-Related Safety Hazards , 2017 .

[29]  Mani Golparvar-Fard,et al.  Appearance-based material classification for monitoring of operation-level construction progress using 4D BIM and site photologs , 2015 .

[30]  Man-Woo Park,et al.  Hardhat-Wearing Detection for Enhancing On-Site Safety of Construction Workers , 2015 .

[31]  Sean M Whitaker,et al.  Safety with access scaffolds: development of a prototype decision aid based on accident analysis. , 2003, Journal of safety research.

[32]  Xiaochun Luo,et al.  A deep learning-based method for detecting non-certified work on construction sites , 2018, Adv. Eng. Informatics.

[33]  Junqing Yu,et al.  Affection arousal based highlight extraction for soccer video , 2013, Multimedia Tools and Applications.

[34]  Oral Büyüköztürk,et al.  Deep Learning‐Based Crack Damage Detection Using Convolutional Neural Networks , 2017, Comput. Aided Civ. Infrastructure Eng..

[35]  Patricio A. Vela,et al.  Construction performance monitoring via still images, time-lapse photos, and video streams: Now, tomorrow, and the future , 2015, Adv. Eng. Informatics.

[36]  Esther Cheung,et al.  Rapid demountable platform (RDP)--a device for preventing fall from height accidents. , 2012, Accident; analysis and prevention.

[37]  Carlos H. Caldas,et al.  Learning and classifying actions of construction workers and equipment using Bag-of-Video-Feature-Words and Bayesian network models , 2011, Adv. Eng. Informatics.

[38]  Guido L. Geerts,et al.  A design science research methodology and its application to accounting information systems research , 2011, Int. J. Account. Inf. Syst..

[39]  Chia-Fen Chi,et al.  Accident patterns and prevention measures for fatal occupational falls in the construction industry. , 2005, Applied ergonomics.

[40]  Jürgen Schmidhuber,et al.  Deep learning in neural networks: An overview , 2014, Neural Networks.

[41]  Changbum R. Ahn,et al.  Fall risk analysis of construction workers using inertial measurement units: validating the usefulness of the postural stability metrics in construction , 2016 .

[42]  Feniosky Peña-Mora,et al.  Empirical assessment of a RGB-D sensor on motion capture and action recognition for construction worker monitoring , 2013 .

[43]  Liang Lin,et al.  Is Faster R-CNN Doing Well for Pedestrian Detection? , 2016, ECCV.

[44]  Andrew Hale,et al.  Quantified risk assessment for fall from height , 2008 .

[45]  Rafael Sacks,et al.  3D Object Classification Using Geometric Features and Pairwise Relationships , 2018, Comput. Aided Civ. Infrastructure Eng..

[46]  Pete Kines,et al.  Construction workers' falls through roofs: fatal versus serious injuries. , 2002, Journal of safety research.

[47]  Junqing Yu,et al.  Dense 3D reconstruction combining depth and RGB information , 2016, Neurocomputing.

[48]  Charles M. Eastman,et al.  BIM-based fall hazard identification and prevention in construction safety planning , 2015 .

[49]  Juan Carlos Niebles,et al.  Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers , 2013, Adv. Eng. Informatics.

[50]  Fei Dai,et al.  Risk Assessment of Work-Related Musculoskeletal Disorders in Construction: State-of-the-Art Review , 2015 .

[51]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[52]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[53]  Jimmie Hinze,et al.  Analysis of Construction Worker Fall Accidents , 2003 .

[54]  Yang Miang Goh.  Empirical Investigation of the Average Deployment Force of Personal Fall-Arrest Energy Absorbers , 2015 .

[55]  Yantao Yu,et al.  An experimental study of real-time identification of construction workers' unsafe behaviors , 2017 .

[56]  SangUk Han,et al.  Tracking-based 3D human skeleton extraction from stereo video camera toward an on-site safety and ergonomic analysis , 2016 .

[57]  Evangelos A. Yfantis,et al.  Hard-Hat Detection for Construction Safety Visualization , 2015 .

[58]  Cordelia Schmid,et al.  Action Recognition with Improved Trajectories , 2013, 2013 IEEE International Conference on Computer Vision.

[59]  Jie Gong,et al.  Computer Vision-Based Video Interpretation Model for Automated Productivity Analysis of Construction Operations , 2010 .

[60]  Peter E.D. Love,et al.  Integrating mobile Building Information Modelling and Augmented Reality systems: An experimental study , 2018 .

[61]  Ioannis Brilakis,et al.  Continuous localization of construction workers via integration of detection and tracking , 2016 .

[62]  J. Aken.  Management Research as a Design Science: Articulating the Research Products of Mode 2 Knowledge Production in Management , 2005 .

[63]  Xiaochun Luo,et al.  Detecting non-hardhat-use by a deep learning method from far-field surveillance videos , 2018 .

[64]  Charles M. Eastman,et al.  Building Information Modeling (BIM) and Safety: Automatic Safety Checking of Construction Models and Schedules , 2013 .

[65]  Ming-Yu Liu,et al.  Deep Active Learning for Civil Infrastructure Defect Detection and Classification , 2017 .

[66]  P Becker,et al.  Prevention of construction falls by organizational intervention , 2001, Injury prevention : journal of the International Society for Child and Adolescent Injury Prevention.

[67]  Reem Zeibak-Shini,et al.  Towards generation of as-damaged BIM models using laser-scanning and as-built BIM: First estimate of as-damaged locations of reinforced concrete frame members in masonry infill structures , 2016, Adv. Eng. Informatics.