Detecting non-hardhat-use by a deep learning method from far-field surveillance videos

Hardhats are an important safety measure used to protect construction workers from accidents. However, accidents caused in ignorance of wearing hardhats still occur. In order to strengthen the supervision of construction workers to avoid accidents, automatic non-hardhat-use (NHU) detection technology can play an important role. Existing automatic methods of detecting hardhat avoidance are commonly limited to the detection of objects in near-field surveillance videos. This paper proposes the use of a high precision, high speed and widely applicable Faster R-CNN method to detect construction workers' NHU. To evaluate the performance of Faster R-CNN, more than 100,000 construction worker image frames were randomly selected from the far-field surveillance videos of 25 different construction sites over a period of more than a year. The research analyzed various visual conditions of the construction sites and classified image frames according to their visual conditions. The image frames were input into Faster R-CNN according to different visual categories. The experimental results demonstrate that the high precision, high recall and fast speed of the method can effectively detect construction workers' NHU in different construction site conditions, and can facilitate improved safety inspection and supervision.

[1]  B Y Jeong,et al.  Occupational deaths and injuries in the construction industry. , 1998, Applied ergonomics.

[2]  John H. Lewko,et al.  Traumatic brain injuries in the construction industry , 2009, Brain injury.

[3]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Huaizu Jiang,et al.  Face Detection with the Faster R-CNN , 2016, 2017 12th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2017).

[5]  Ryan W. Hoover,et al.  The effect of hardhats on head and neck response to vertical impacts from large construction objects. , 2014, Accident; analysis and prevention.

[6]  Man-Woo Park,et al.  Hardhat-Wearing Detection for Enhancing On-Site Safety of Construction Workers , 2015 .

[7]  Hakil Kim,et al.  On-road object detection using deep neural network , 2016, 2016 IEEE International Conference on Consumer Electronics-Asia (ICCE-Asia).

[8]  P Susi,et al.  Ergonomics and construction: a review of potential hazards in new construction. , 1994, American Industrial Hygiene Association journal.

[9]  Srinivas Konda,et al.  Fatal traumatic brain injuries in the construction industry, 2003-2010. , 2016, American journal of industrial medicine.

[10]  Bill Triggs,et al.  Histograms of oriented gradients for human detection , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[11]  Jimmie Hinze,et al.  Analysis of Construction Worker Fall Accidents , 2003 .

[12]  J. Aggarwal,et al.  Recognizing human action from a far field of view , 2009, 2009 Workshop on Motion and Video Computing (WMVC).

[13]  Jimmie Hinze,et al.  Leading indicators of construction safety performance , 2013 .

[14]  Nicoletta Cavazza,et al.  Effects of safety climate on safety norm violations: exploring the mediating role of attitudinal ambivalence toward personal protective equipment. , 2009, Journal of safety research.

[15]  Manoranjan Paul,et al.  Human detection in surveillance videos and its applications - a review , 2013, EURASIP J. Adv. Signal Process..

[16]  Neil A. Dodgson,et al.  Proceedings Ninth IEEE International Conference on Computer Vision , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[17]  Feniosky Peña-Mora,et al.  Comparative Study of Motion Features for Similarity-Based Modeling and Classification of Unsafe Actions in Construction , 2014, J. Comput. Civ. Eng..

[18]  Jochen Teizer,et al.  Mobile passive Radio Frequency Identification (RFID) portal for automated and rapid control of Personal Protective Equipment (PPE) on construction sites , 2013 .

[19]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[20]  Dumitru Erhan,et al.  Deep Neural Networks for Object Detection , 2013, NIPS.

[21]  SangUk Han,et al.  A vision-based motion capture and recognition framework for behavior-based safety management , 2013 .

[22]  Dapeng Hong,et al.  Silhouette-Based On-Site Human Action Recognition in Single-View Video , 2016 .

[23]  David A. Lombardi,et al.  Factors influencing worker use of personal protective eyewear. , 2009, Accident; analysis and prevention.

[24]  Ross B. Girshick,et al.  Fast R-CNN , 2015, 1504.08083.

[25]  Evangelos A. Yfantis,et al.  Hard-Hat Detection for Construction Safety Visualization , 2015 .

[26]  Jian Sun,et al.  Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Mohan M. Trivedi,et al.  Driver hand localization and grasp analysis: A vision-based real-time approach , 2016, 2016 IEEE 19th International Conference on Intelligent Transportation Systems (ITSC).

[28]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Yanfang Ye,et al.  Automatic Detection of Helmet Uses for Construction Safety , 2016, 2016 IEEE/WIC/ACM International Conference on Web Intelligence Workshops (WIW).

[30]  Srinivas Konda Mph,et al.  Fatal traumatic brain injuries in the construction industry, 2003−2010 , 2016 .

[31]  Kathryn Mearns,et al.  Measuring safety climate: identifying the common features☆ , 2000 .

[32]  Qiu Guan,et al.  Cancer Cells Detection in Phase-Contrast Microscopy Images Based on Faster R-CNN , 2016, 2016 9th International Symposium on Computational Intelligence and Design (ISCID).

[33]  Xu Wang,et al.  Pedestrian Detection for Transformer Substation Based on Gaussian Mixture Model and YOLO , 2016, 2016 8th International Conference on Intelligent Human-Machine Systems and Cybernetics (IHMSC).

[34]  Ngai-Man Cheung,et al.  Image-based vehicle analysis using deep neural network: A systematic study , 2016, 2016 IEEE International Conference on Digital Signal Processing (DSP).

[35]  Ruel R. Cabahug A Survey on the Implementation of Safety Standards of On-going Construction Projects in Cagayan de Oro City, Philippines , 2014 .

[36]  Heng Li,et al.  Automated PPE Misuse Identification and Assessment for Safety Performance Enhancement , 2015 .

[37]  Vivian W. Y Tam,et al.  A feasibility study of introducing chin straps of safety helmets as a statutory requirement in Hong Kong construction industry , 2014 .

[38]  F Akbar-Khanzadeh Factors contributing to discomfort or dissatisfaction as a result of wearing personal protective equipment. , 1998, Journal of human ergology.

[39]  Man-Woo Park,et al.  Automated monitoring of hardhats wearing for onsite safety enhancement , 2015 .

[40]  Xiaochun Luo,et al.  Investigation of the causality patterns of non-helmet use behavior of construction workers , 2017 .

[41]  Mark Everingham,et al.  Implicit color segmentation features for pedestrian and object detection , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[42]  JoonOh Seo,et al.  A Stereo Vision-based Approach to Marker-less Motion Capture for On-Site Kinematic Modeling of Construction Worker Tasks , 2014 .

[43]  Wael Badawy,et al.  Hard hat detection in video sequences based on face features, motion and color information , 2011, 2011 3rd International Conference on Computer Research and Development.

[44]  Jochen Teizer,et al.  Real-time construction worker posture analysis for ergonomics training , 2012, Adv. Eng. Informatics.

[45]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[46]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[47]  Rong Jin,et al.  Multiple Kernel Learning for Visual Object Recognition: A Review , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Berardo Naticchia,et al.  A monitoring system for real-time interference control on large construction sites , 2013 .

[49]  Tiago M. Fernández-Caramés,et al.  Real-time personal protective equipment monitoring system , 2012, Comput. Commun..

[50]  William T. Freeman,et al.  Latent hierarchical structural learning for object detection , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[51]  David A. McAllester,et al.  Object Detection with Discriminatively Trained Part Based Models , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[52]  Rogério Schmidt Feris,et al.  Robust Detection of Abandoned and Removed Objects in Complex Surveillance Videos , 2011, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[53]  Wei Liu,et al.  SSD: Single Shot MultiBox Detector , 2015, ECCV.