Real-time simulation of construction workers using combined human body and hand tracking for robotic construction worker system

Abstract Construction is an inherently less safe sector than other sectors because it exposes workers to harsh and dangerous working environments. The nature of the construction industry results in a comparatively high incidence of serious injuries and death caused by falls from a height, musculoskeletal disorders and being struck by objects. This paper presents a new concept that can tackle this problem in the future. The central hypothesis of this study is that it is possible to eliminate injuries if we move the human construction worker off-site and remotely link his/her motions to a Robotic Construction Worker (RCW) on-site. As a first steppingstone towards this ultimate goal, two systems essential for the RCW were developed in this study. First, a novel system that combines 3D body and hand position tracking was developed to capture the movements of human construction worker. This combination of tracking enables the capture of changes in the orientations and articulations of the entire human body. Second, a real-time simulation system that connects a human construction worker off-site to a virtual RCW was developed to demonstrate the proposed concept in a variety of construction scenarios. The simulation results demonstrate the future viability of the RCW concept and indicate the promise of this system for eliminating the health and safety risks faced by human construction workers.

[1]  Rudi Stouffs,et al.  Construction safety risk drivers: A BIM approach , 2016 .

[2]  Tao Cheng,et al.  Data Fusion of Real-Time Location Sensing and Physiological Status Monitoring for Ergonomics Analysis of Construction Workers , 2013, J. Comput. Civ. Eng..

[3]  Mohammad H. Mahoor,et al.  Development of a body joint angle measurement system using IMU sensors , 2011, 2011 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[4]  SangHyun Lee,et al.  Computer vision techniques for construction safety and health monitoring , 2015, Adv. Eng. Informatics.

[5]  Tang Xinxing,et al.  Tele-operation Construction Robot Control System with Virtual Reality , 2009 .

[6]  Frédéric Bosché,et al.  Musculoskeletal disorders in construction: A review and a novel system for activity tracking with body area network. , 2016, Applied ergonomics.

[7]  Changbum R. Ahn,et al.  Fall risk analysis of construction workers using inertial measurement units: validating the usefulness of the postural stability metrics in construction , 2016 .

[8]  Giovanni C. Migliaccio,et al.  Wearable sensors for monitoring on-duty and off-duty worker physiological status and activities in construction , 2017 .

[9]  Zhenhua Li,et al.  Registration of the Cone Beam CT and Blue-Ray Scanned Dental Model Based on the Improved ICP Algorithm , 2014, Int. J. Biomed. Imaging.

[10]  Jochen Teizer,et al.  Real-time construction worker posture analysis for ergonomics training , 2012, Adv. Eng. Informatics.

[11]  Arto Kiviniemi,et al.  A review of risk management through BIM and BIM-related technologies , 2017 .

[12]  Andrew W. Fitzgibbon,et al.  Accurate, Robust, and Flexible Real-time Hand Tracking , 2015, CHI.

[13]  Caterina Rizzi,et al.  A virtual environment to emulate tailor’s work , 2016 .

[14]  LeeSangHyun,et al.  Computer vision techniques for construction safety and health monitoring , 2015 .

[15]  Eric M. Wetzel,et al.  The use of a BIM-based framework to support safe facility management processes , 2015 .

[16]  Ren-Jye Dzeng,et al.  Accelerometer-based fall-portent detection algorithm for construction tiling operation , 2017 .

[17]  Jingdao Chen,et al.  A framework for real-time pro-active safety assistance for mobile crane lifting operations , 2016 .

[18]  Rafael Sacks,et al.  Construction Job Safety Analysis , 2010 .

[19]  Caterina Rizzi,et al.  Mixed Reality to Design Lower Limb Prosthesis , 2015 .

[20]  Yale Song,et al.  Tracking body and hands for gesture recognition: NATOPS aircraft handling signals database , 2011, Face and Gesture 2011.

[21]  Ari Paasio,et al.  Accelerometer-Based Method for Extracting Respiratory and Cardiac Gating Information for Dual Gating during Nuclear Medicine Imaging , 2014, Int. J. Biomed. Imaging.

[22]  Tuukka M. Takala,et al.  Reality-based User Interface System (RUIS) , 2011, 3DUI.

[23]  Antonis A. Argyros,et al.  Efficient model-based 3D tracking of hand articulations using Kinect , 2011, BMVC.

[24]  John B. Shoven,et al.  I , Edinburgh Medical and Surgical Journal.

[25]  Feniosky Peña-Mora,et al.  Vision-Based Detection of Unsafe Actions of a Construction Worker: Case Study of Ladder Climbing , 2013, J. Comput. Civ. Eng..

[26]  Charles M. Eastman,et al.  Building Information Modeling (BIM) and Safety: Automatic Safety Checking of Construction Models and Schedules , 2013 .