Ergonomic posture recognition using 3D view-invariant features from single ordinary camera

Abstract Manual construction tasks are physically demanding, requiring prolonged awkward postures that can cause pain and injury. Person posture recognition (PPR) is essential in postural ergonomic hazard assessment. This paper proposed an ergonomic posture recognition method using 3D view-invariant features from a single 2D camera that is non-intrusive and widely installed on construction sites. Based on the detected 2D skeletons, view-invariant relative 3D joint position (R3DJP) and joint angle are extracted as classification features by employing a multi-stage convolutional nerual network (CNN) architecture, so that the learned classifier is not sensitive to camera viewpoints. Three posture classifiers regarding arms, back, and legs are trained, so that they can be simultaneously classified in one video frame. The posture recognition accuracies of three body parts are 98.6%, 99.5%, 99.8%, respectively. For generalization ability, the relevant accuracies are 94.9%, 93.9%, 94.6%, respectively. Both the classification accuracy and generalization ability of the method outperform previous vision-based methods in construction. The proposed method enables reliable and accurate postural ergonomic assessment for improving construction workers' safety and healthy.

[1]  Thi-Lan Le,et al.  Human posture recognition using human skeleton provided by Kinect , 2013, 2013 International Conference on Computing, Management and Telecommunications (ComManTel).

[2]  Tzu-Hsien Lee,et al.  Analysis of Working Postures at a Construction Site Using the OWAS Method , 2013, International journal of occupational safety and ergonomics : JOSE.

[3]  Karlheinz Schaub.  Ergonomics of Manual Handling - Part 1: Lifting and Carrying , 2006 .

[4]  Hong Zhang,et al.  Wearable IMU-based real-time motion warning system for construction workers' musculoskeletal disorders prevention , 2017 .

[5]  Wen Yi,et al.  Health Profile of Construction Workers in Hong Kong , 2016, International journal of environmental research and public health.

[6]  J H van Dieën,et al.  Stoop or squat: a review of biomechanical studies on lifting technique. , 1999, Clinical biomechanics.

[7]  Sang D. Choi,et al.  Work-related musculoskeletal disorders among construction workers in the United States from 1992 to 2014 , 2016, Occupational and Environmental Medicine.

[8]  Kourosh Khoshelham,et al.  Accuracy analysis of kinect depth data , 2012 .

[9]  Sakiko Oyama,et al.  Accuracy and repeatability of an inertial measurement unit system for field-based occupational studies , 2016, Ergonomics.

[10]  O Karhu,et al.  Correcting working postures in industry: A practical method for analysis. , 1977, Applied ergonomics.

[11]  J Dul,et al.  International standards on working postures and movements ISO 11226 and EN 1005-4 , 2007, Ergonomics.

[12]  Xiaochun Luo,et al.  Detecting non-hardhat-use by a deep learning method from far-field surveillance videos , 2018 .

[13]  Cordelia Schmid,et al.  Learning realistic human actions from movies , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Hassan Foroosh,et al.  View-Invariant Action Recognition from Point Triplets , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Raymond E. Levitt,et al.  Construction safety management , 1987 .

[16]  Ze-Nian Li,et al.  Successive Convex Matching for Action Detection , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[17]  Jun Li,et al.  Deeply Learned View-Invariant Features for Cross-View Action Recognition , 2017, IEEE Transactions on Image Processing.

[18]  Ken-Yu Lin,et al.  An evaluation of wearable sensors and their placements for analyzing construction worker's trunk posture in laboratory conditions. , 2017, Applied ergonomics.

[19]  Leon Straker,et al.  In vivo laboratory validation of the physiometer: a measurement system for long-term recording of posture and movements in the workplace , 2010, Ergonomics.

[20]  Ronald Poppe,et al.  A survey on vision-based human action recognition , 2010, Image Vis. Comput..

[21]  Feniosky Peña-Mora,et al.  Comparative Study of Motion Features for Similarity-Based Modeling and Classification of Unsafe Actions in Construction , 2014, J. Comput. Civ. Eng..

[22]  Lourdes Agapito,et al.  Lifting from the Deep: Convolutional 3D Pose Estimation from a Single Image , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  L McAtamney,et al.  RULA: a survey method for the investigation of work-related upper limb disorders. , 1993, Applied ergonomics.

[24]  Mohammad Nahangi,et al.  Level-of-Expertise Classification for Identifying Safe and Productive Masons , 2017 .

[25]  Mubarak Shah,et al.  Recognizing human actions using multiple features , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Thomas J. Armstrong,et al.  Motion Data-Driven Biomechanical Analysis during Construction Tasks on Sites , 2015 .

[27]  Patricio A. Vela,et al.  Construction performance monitoring via still images, time-lapse photos, and video streams: Now, tomorrow, and the future , 2015, Adv. Eng. Informatics.

[28]  Lourdes Agapito,et al.  Learning a Manifold as an Atlas , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Ayman Habib,et al.  Application of Microsoft Kinect Sensor for Tracking Construction Workers , 2012 .

[30]  Feniosky Peña-Mora,et al.  Empirical assessment of a RGB-D sensor on motion capture and action recognition for construction worker monitoring , 2013 .

[31]  Karlheinz Schaub,et al.  ISO/FDIS 11226: Evaluation of Static Working Postures , 2000 .

[32]  Feniosky Peña-Mora,et al.  Application of dimension reduction techniques for motion recognition: Construction worker behavior monitoring , 2011 .

[33]  Yantao Yu,et al.  Visualization technology-based construction safety management: A review , 2017 .

[34]  Feniosky Peña-Mora,et al.  Vision-Based Detection of Unsafe Actions of a Construction Worker: Case Study of Ladder Climbing , 2013, J. Comput. Civ. Eng..

[35]  Hans-Peter Seidel,et al.  VNect , 2017, ACM Trans. Graph..

[36]  Cristian Sminchisescu,et al.  Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[37]  Heng Li,et al.  Identification of Biomechanical Risk Factors for the Development of Lower-Back Disorders during Manual Rebar Tying , 2017 .

[38]  Pascal Fua,et al.  Learning to Fuse 2D and 3D Image Cues for Monocular Body Pose Estimation , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[39]  J.B. Burns,et al.  View Variation of Point-Set and Line-Segment Features , 1993, IEEE Trans. Pattern Anal. Mach. Intell..

[40]  Jake K. Aggarwal,et al.  View invariant human action recognition using histograms of 3D joints , 2012, 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[41]  H Laitinen,et al.  The validity of the TR safety observation method on building construction. , 1999, Accident; analysis and prevention.

[42]  Jun Qiu,et al.  Construction worker's awkward posture recognition through supervised motion tensor decomposition , 2017 .

[43]  SangUk Han,et al.  An Automated Biomechanical Simulation Approach to Ergonomic Job Analysis for Workplace Design , 2015 .

[44]  Mohamed Al-Hussein,et al.  3D Visualization-Based Ergonomic Risk Assessment and Work Modification Framework and Its Validation for a Lifting Task , 2018 .

[45]  Mubarak Shah,et al.  Human Action Recognition in Videos Using Kinematic Features and Multiple Instance Learning , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[46]  Hong Zhang,et al.  Development of ergonomic posture recognition technique based on 2D ordinary camera for construction hazard prevention through view-invariant features in 2D skeleton motion , 2017, Adv. Eng. Informatics.

[47]  Toby Sharp,et al.  Real-time human pose recognition in parts from single depth images , 2011, CVPR.

[48]  Fiora Pirri,et al.  Bayesian Image Based 3D Pose Estimation , 2016, ECCV.

[49]  Jochen Teizer,et al.  Real-time construction worker posture analysis for ergonomics training , 2012, Adv. Eng. Informatics.

[50]  Varun Ramakrishna,et al.  Convolutional Pose Machines , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[51]  SangHyun Lee,et al.  Automated Postural Ergonomic Assessment Using a Computer Vision-Based Posture Classification , 2016 .