Machine Learning-Based Risk Analysis for Construction Worker Safety from Ubiquitous Site Photos and Videos

AbstractThis paper proposes a new method for single-worker severity level prediction from already collected site images and video clips. Onsite safety observers often assess workers’ severity level...

[1]  Peter E.D. Love,et al.  Computer vision applications in construction safety assurance , 2020 .

[2]  Mani Golparvar-Fard,et al.  Real-Time and Automated Recognition and 2D Tracking of Construction Workers and Equipment from Site Video Streams , 2012 .

[3]  Xiaochun Luo,et al.  Vision-based detection and visualization of dynamic workspaces , 2019, Automation in Construction.

[4]  L McAtamney,et al.  RULA: a survey method for the investigation of work-related upper limb disorders. , 1993, Applied ergonomics.

[5]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Nicola Paltrinieri,et al.  Learning about risk: Machine learning for risk assessment , 2019, Safety Science.

[7]  Patricio A. Vela,et al.  Construction performance monitoring via still images, time-lapse photos, and video streams: Now, tomorrow, and the future , 2015, Adv. Eng. Informatics.

[8]  Mani Golparvar-Fard,et al.  Potential of big visual data and building information modeling for construction performance analytics: An exploratory study , 2017 .

[9]  Silvio Savarese,et al.  Structural-RNN: Deep Learning on Spatio-Temporal Graphs , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[10]  O Karhu,et al.  Correcting working postures in industry: A practical method for analysis. , 1977, Applied ergonomics.

[11]  Jixiu Wu,et al.  Automatic detection of hardhats worn by construction personnel: A deep learning approach and benchmark dataset , 2019, Automation in Construction.

[12]  Brenda McCabe,et al.  Automated Visual Recognition of Dump Trucks in Construction Videos , 2012, J. Comput. Civ. Eng..

[13]  Juan Carlos Niebles,et al.  Vision-based action recognition of earthmoving equipment using spatio-temporal features and support vector machine classifiers , 2013, Adv. Eng. Informatics.

[14]  Rafael Sacks,et al.  Construction Job Safety Analysis , 2010 .

[15]  SangHyeok Han,et al.  Monocular Vision-Based Framework for Biomechanical Analysis or Ergonomic Posture Assessment in Modular Construction , 2020, J. Comput. Civ. Eng..

[16]  Hong Zhang,et al.  Personalized method for self-management of trunk postural ergonomic hazards in construction rebar ironwork , 2018, Adv. Eng. Informatics.

[17]  John A. Gambatese,et al.  Population and Initial Validation of a Formal Model for Construction Safety Risk Management , 2010 .

[18]  Amir H. Behzadan,et al.  Deep learning for site safety: Real-time detection of personal protective equipment , 2020 .

[19]  Peter E.D. Love,et al.  A deep hybrid learning model to detect unsafe behavior: Integrating convolution neural networks and long short-term memory , 2018 .

[20]  Kaiming He,et al.  Mask R-CNN , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[21]  Mingyuan Zhang,et al.  A critical review of vision-based occupational health and safety monitoring of construction site workers , 2020, Safety Science.

[22]  L Punnett,et al.  PATH: a work sampling-based approach to ergonomic job analysis for construction and other non-repetitive work. , 1996, Applied ergonomics.

[23]  Feniosky Peña-Mora,et al.  Vision-Based Detection of Unsafe Actions of a Construction Worker: Case Study of Ladder Climbing , 2013, J. Comput. Civ. Eng..

[24]  Mi Zhu,et al.  Recognition of High-Risk Scenarios in Building Construction Based on Image Semantics , 2020, J. Comput. Civ. Eng..

[25]  Xiaochun Luo,et al.  Capturing and Understanding Workers’ Activities in Far‐Field Surveillance Videos with Deep Action Recognition and Bayesian Nonparametric Learning , 2018, Comput. Aided Civ. Infrastructure Eng..

[26]  Peter E.D. Love,et al.  Falls from heights: A computer vision-based approach for safety harness detection , 2018, Automation in Construction.

[27]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[28]  Svein J. Knapskog,et al.  Real-Time Risk Assessment with Network Sensors and Intrusion Detection Systems , 2005, CIS.

[29]  Heng Li,et al.  Estimating Worker-Centric 3D Spatial Crowdedness for Construction Safety Management Using a Single 2D Camera , 2019, J. Comput. Civ. Eng..

[30]  Zhongke Shi,et al.  Vision-based action recognition of construction workers using dense trajectories , 2016, Adv. Eng. Informatics.

[31]  SangHyun Lee,et al.  Computer vision techniques for construction safety and health monitoring , 2015, Adv. Eng. Informatics.

[32]  Ioannis Brilakis,et al.  Construction worker detection in video frames for initializing vision trackers , 2012 .

[33]  Mani Golparvar-Fard,et al.  Vision-based workface assessment using depth images for activity analysis of interior construction operations , 2014 .

[34]  Dario Pavllo,et al.  3D Human Pose Estimation in Video With Temporal Convolutions and Semi-Supervised Training , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Mani Golparvar-Fard,et al.  Human-object interaction recognition for automatic construction site safety inspection , 2020 .

[36]  M. Golparvar-Fard,et al.  Vision-Based Construction Worker Activity Analysis Informed by Body Posture , 2020, J. Comput. Civ. Eng..

[37]  S Hignett,et al.  Rapid entire body assessment (REBA). , 2000, Applied ergonomics.

[38]  Heng Li,et al.  Computer vision aided inspection on falling prevention measures for steeplejacks in an aerial environment , 2018, Automation in Construction.

[39]  Xiaochun Luo,et al.  Towards efficient and objective work sampling: Recognizing workers' activities in site surveillance videos with two-stream convolutional networks , 2018, Automation in Construction.

[40]  Yang Miang Goh,et al.  Safety leading indicators for construction sites: A machine learning approach , 2018, Automation in Construction.

[41]  Xiaochun Luo,et al.  Joint-Level Vision-Based Ergonomic Assessment Tool for Construction Workers , 2019, Journal of Construction Engineering and Management.

[42]  Peter E.D. Love,et al.  Convolutional neural networks: Computer vision-based workforce activity assessment in construction , 2018, Automation in Construction.

[43]  Man-Woo Park,et al.  Hardhat-Wearing Detection for Enhancing On-Site Safety of Construction Workers , 2015 .

[44]  Mani Golparvar-Fard,et al.  A Framework for Model-Driven Acquisition and Analytics of Visual Data Using UAVs for Automated Construction Progress Monitoring , 2015 .

[45]  Jochen Teizer,et al.  Real-time construction worker posture analysis for ergonomics training , 2012, Adv. Eng. Informatics.

[46]  Evangelos A. Yfantis,et al.  Hard-Hat Detection for Construction Safety Visualization , 2015 .

[47]  Tao Cheng,et al.  Real-time resource location data collection and visualization technology for construction safety and activity monitoring applications , 2013 .

[48]  Claudio Delrieux,et al.  Ergonomic risk assessment based on computer vision and machine learning , 2020, Comput. Ind. Eng..

[49]  Rafael Sacks,et al.  Spatial and Temporal Exposure to Safety Hazards in Construction , 2009 .

[50]  Xiaochun Luo,et al.  Detecting non-hardhat-use by a deep learning method from far-field surveillance videos , 2018 .

[51]  Hubo Cai,et al.  Two-step long short-term memory method for identifying construction activities through positional and attentional cues , 2019, Automation in Construction.

[52]  SangUk Han,et al.  Tracking-based 3D human skeleton extraction from stereo video camera toward an on-site safety and ergonomic analysis , 2016 .

[53]  Heng Li,et al.  Ergonomic posture recognition using 3D view-invariant features from single ordinary camera , 2018, Automation in Construction.

[54]  Jacob Cohen,et al.  The Equivalence of Weighted Kappa and the Intraclass Correlation Coefficient as Measures of Reliability , 1973 .

[55]  Kaiming He,et al.  Feature Pyramid Networks for Object Detection , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[56]  Yuxuan Wang,et al.  Onsite video mining for construction hazards identification with visual relationships , 2019, Adv. Eng. Informatics.

[57]  Rafiq M Choudhry,et al.  Behavior-based safety on construction sites: a case study. , 2014, Accident; analysis and prevention.

[58]  Gerardo Cupido,et al.  Cognitive Approach to Construction Safety: Task Demand-Capability Model , 2009 .

[59]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.