Motion Capture Technology in Industrial Applications: A Systematic Review

The rapid technological advancements of Industry 4.0 have opened up new vectors for novel industrial processes that require advanced sensing solutions for their realization. Motion capture (MoCap) sensors, such as visual cameras and inertial measurement units (IMUs), are frequently adopted in industrial settings to support solutions in robotics, additive manufacturing, teleworking and human safety. This review synthesizes and evaluates studies investigating the use of MoCap technologies in industry-related research. A search was performed in the Embase, Scopus, Web of Science and Google Scholar. Only studies in English, from 2015 onwards, on primary and secondary industrial applications were considered. The quality of the articles was appraised with the AXIS tool. Studies were categorized based on type of used sensors, beneficiary industry sector, and type of application. Study characteristics, key methods and findings were also summarized. In total, 1682 records were identified, and 59 were included in this review. Twenty-one and 38 studies were assessed as being prone to medium and low risks of bias, respectively. Camera-based sensors and IMUs were used in 40% and 70% of the studies, respectively. Construction (30.5%), robotics (15.3%) and automotive (10.2%) were the most researched industry sectors, whilst health and safety (64.4%) and the improvement of industrial processes or products (17%) were the most targeted applications. Inertial sensors were the first choice for industrial MoCap applications. Camera-based MoCap systems performed better in robotic applications, but camera obstructions caused by workers and machinery was the most challenging issue. Advancements in machine learning algorithms have been shown to increase the capabilities of MoCap systems in applications such as activity and fatigue detection as well as tool condition monitoring and object recognition.

[1]  Enrique Valero,et al.  Analysis of construction trade worker body motions using a wearable and wireless motion sensor network , 2017 .

[2]  Carlo Alberto Avizzano,et al.  A novel wearable system for the online assessment of risk for biomechanical load in repetitive efforts , 2016 .

[3]  Chris Bregler,et al.  Motion Capture Technology for Entertainment , 2007 .

[4]  D. Roetenberg,et al.  Xsens MVN: Full 6DOF Human Motion Tracking Using Miniature Inertial Sensors , 2009 .

[5]  Jon Clarke,et al.  Identifying car ingress movement strategies before and after total knee replacement , 2020, International biomechanics.

[6]  Günther Seliger,et al.  Motion Tracking Applied in Assembly for Worker Training in different Locations , 2016 .

[7]  Francesco Caputo,et al.  A Human Postures Inertial Tracking System for Ergonomic Assessments , 2018, Advances in Intelligent Systems and Computing.

[8]  Nicolas Vuillerme,et al.  Trunk kinematics and low back pain during pruning among vineyard workers—A field study at the Chateau Larose-Trintaudon , 2017, PloS one.

[9]  Rachel S Dean,et al.  Development of a critical appraisal tool to assess the quality of cross-sectional studies (AXIS) , 2016, BMJ Open.

[10]  Fabian Mueller,et al.  Intuitive Welding Robot Programming via Motion Capture and Augmented Reality , 2019, IFAC-PapersOnLine.

[11]  Mark Burgess,et al.  Remote real-time collaboration through synchronous exchange of digitised human-workpiece interactions , 2017, Future Gener. Comput. Syst..

[12]  Hyunsoo Kim,et al.  Identifying Safety Hazards Using Collective Bodily Responses of Workers , 2017 .

[13]  Hiroyuki Sawada,et al.  Retrofitting old CNC turning with an accelerometer at a remote location towards Industry 4.0 , 2019, Manufacturing Letters.

[14]  Mingjie Dong,et al.  A new positioning method for remotely operated vehicle of the nuclear power plant , 2020, Ind. Robot.

[15]  Andreas Holtermann,et al.  Objectively measured physical activity and 12-month trajectories of neck–shoulder pain in workers: A prospective study in DPHACTO , 2017, Scandinavian journal of public health.

[16]  Francesco Caputo,et al.  Integrated wearable devices for evaluating the biomechanical overload in manufacturing , 2019, 2019 II Workshop on Metrology for Industry 4.0 and IoT (MetroInd4.0&IoT).

[17]  Jingdao Chen,et al.  Evaluation of Machine Learning Algorithms for Worker’s Motion Recognition Using Motion Sensors , 2019, Computing in Civil Engineering 2019.

[18]  M. M. Reijne,et al.  Accuracy of human motion capture systems for sport applications; state-of-the-art review , 2018, European journal of sport science.

[19]  Mehmet C. Vuran,et al.  Collective sensing of workers’ gait patterns to identify fall hazards in construction , 2017 .

[20]  Youngjib Ham,et al.  Motion and Visual Data-Driven Distant Object Localization for Field Reporting , 2018 .

[21]  JoonOh Seo,et al.  A comparative study of in-field motion capture approaches for body kinematics measurement in construction , 2017, Robotica.

[22]  Stefan Mengel,et al.  Using Marker-less Motion Capture Systems for Walk Path Analysis in Paced Assembly Flow Lines , 2016 .

[23]  Ju Jin,et al.  The adaptive Kalman filter based on fuzzy logic for inertial motion capture system , 2014 .

[24]  Jack P Callaghan,et al.  Biomechanical investigation of prolonged driving in an ergonomically designed truck seat prototype , 2018, Ergonomics.

[25]  Xiaowei Luo,et al.  A proactive workers' safety risk evaluation framework based on position and posture data fusion , 2019, Automation in Construction.

[26]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]  T. Seeberg,et al.  Towards a wearable sensor system for continuous occupational cold stress assessment , 2018, Industrial health.

[28]  François Charpillet,et al.  Activity Recognition With Multiple Wearable Sensors for Industrial Applications , 2018 .

[29]  Lorenzo Fiori,et al.  Biomechanical Load Evaluation by Means of Wearable Devices in Industrial Environments: An Inertial Motion Capture System and sEMG Based Protocol , 2018 .

[30]  Jeongsoo Lee,et al.  Development of a Wearable Sensing Glove for Measuring the Motion of Fingers Using Linear Potentiometers and Flexible Wires , 2015, IEEE Transactions on Industrial Informatics.

[31]  Changbum R. Ahn,et al.  Comprehensive Fall-Risk Assessment of Construction Workers Using Inertial Measurement Units: Validation of the Gait-Stability Metric to Assess the Fall Risk of Iron Workers , 2016, J. Comput. Civ. Eng..

[32]  Sarah R. Fletcher,et al.  A study to trial the use of inertial non-optical motion capture for ergonomic analysis of manufacturing work , 2018 .

[33]  J H van Dieën,et al.  Continuous ambulatory hand force monitoring during manual materials handling using instrumented force shoes and an inertial motion capture suit. , 2017, Journal of biomechanics.

[34]  Ehsan T. Esfahani,et al.  A machine learning approach to detect changes in gait parameters following a fatiguing occupational task , 2018, Ergonomics.

[35]  Lorenzo Fiori,et al.  Biomechanical Overload Evaluation in Manufacturing: A Novel Approach with sEMG and Inertial Motion Capture Integration , 2018 .

[36]  Zahra Sedighi Maman,et al.  A data-driven approach to modeling physical fatigue in the workplace using wearable sensors. , 2017, Applied ergonomics.

[37]  Tilak Dutta,et al.  Evaluation of the Kinect™ sensor for 3-D kinematic measurement in the workplace. , 2012, Applied ergonomics.

[38]  Changbum R. Ahn,et al.  Fall risk analysis of construction workers using inertial measurement units: validating the usefulness of the postural stability metrics in construction , 2016 .

[39]  Mehmet C. Vuran,et al.  Threshold-Based Approach to Detect Near-Miss Falls of Iron Workers Using Inertial Measurement Units , 2015 .

[40]  D. Moher,et al.  Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. , 2010, International journal of surgery.

[41]  Xu Xu,et al.  A computer vision based method for 3D posture estimation of symmetrical lifting. , 2018, Journal of biomechanics.

[42]  Eugenio Merino,et al.  Ergonomic evaluation of the musculoskeletal risks in a banana harvesting activity through qualitative and quantitative measures, with emphasis on motion capture (Xsens) and EMG , 2019, International Journal of Industrial Ergonomics.

[43]  Hossein Mousavi Hondori,et al.  A Review on Technical and Clinical Impact of Microsoft Kinect on Physical Therapy and Rehabilitation , 2014, Journal of medical engineering.

[44]  Angelo Cenedese,et al.  A machine learning approach for gesture recognition with a lensless smart sensor system , 2018, 2018 IEEE 15th International Conference on Wearable and Implantable Body Sensor Networks (BSN).

[45]  Mauro Gamberi,et al.  Motion Analysis System for the digitalization and assessment of manual manufacturing and assembly processes , 2018 .

[46]  Jörg Krüger,et al.  Automated vision-based live ergonomics analysis in assembly operations , 2015 .

[47]  Agathoniki Trigoni,et al.  Tracking People in Highly Dynamic Industrial Environments , 2017, IEEE Transactions on Mobile Computing.

[48]  Alberto Villa,et al.  Monitoring the human posture in industrial environment: A feasibility study , 2018, 2018 IEEE Sensors Applications Symposium (SAS).

[49]  Andrea Maria Zanchettin,et al.  Trajectory generation algorithm for safe human-robot collaboration based on multiple depth sensor measurements , 2018, Mechatronics.

[50]  Chris Bregler,et al.  Motion Capture Technology for Entertainment [In the Spotlight] , 2007, IEEE Signal Processing Magazine.

[51]  Weidong Geng,et al.  Reuse of Motion Capture Data in Animation: A Review , 2003, ICCSA.

[52]  Ashutosh Tiwari,et al.  Digitisation of a moving assembly operation using multiple depth imaging sensors , 2016 .

[53]  Colleen A. Brents,et al.  Low Back Biomechanics of Keg Handling Using Inertial Measurement Units , 2018, Advances in Intelligent Systems and Computing.

[54]  Mehmet C. Vuran,et al.  Semi-supervised near-miss fall detection for ironworkers with a wearable inertial measurement unit , 2016 .

[55]  Mohammed Hossny,et al.  Skeleton-free RULA ergonomic assessment using Kinect sensors , 2017, Intell. Decis. Technol..

[56]  Gordon Dobie,et al.  Mobile robot positioning using accelerometers for pipe inspection , 2019 .

[57]  Douglas E. Dow,et al.  Control for Camera of a Telerobotic Human Computer Interface , 2015 .

[58]  Reza Akhavian,et al.  Ergonomic analysis of construction worker's body postures using wearable mobile sensors. , 2017, Applied ergonomics.

[59]  Antonio Frisoli,et al.  An emg-based robotic hand exoskeleton for bilateral training of grasp , 2013, 2013 World Haptics Conference (WHC).

[60]  Salil S. Kanhere,et al.  WashInDepth: Lightweight Hand Wash Monitor Using Depth Sensor , 2016, MobiQuitous.

[61]  Chris Button,et al.  A Review of Vision-Based Motion Analysis in Sport , 2008, Sports medicine.

[62]  M. Rosenberger,et al.  Efficient 3D object tracking approach based on convolutional neural network and Monte Carlo algorithms used for a pick and place robot , 2019, Other Conferences.

[63]  Mauro Gamberi,et al.  Motion Analysis System (MAS) for production and ergonomics assessment in the manufacturing processes , 2020, Comput. Ind. Eng..

[64]  Fazel Naghdy,et al.  Motion capture in robotics review , 2009, 2009 IEEE International Conference on Control and Automation.

[65]  Zhengyou Zhang,et al.  Microsoft Kinect Sensor and Its Effect , 2012, IEEE Multim..

[66]  John Barton,et al.  Predicting Three-Dimensional Ground Reaction Forces in Running by Using Artificial Neural Networks and Lower Body Kinematics , 2019, IEEE Access.

[67]  Brendan O'Flynn,et al.  3D ranging and tracking using lensless smart sensors , 2017 .

[68]  Martin Manns,et al.  Real-Time Motion Tracking for Humans and Robots in a Collaborative Assembly Task , 2019, Proceedings.

[69]  Stefano Paolo Pastorelli,et al.  Implementing a Vision-Based Collision Avoidance Algorithm on a UR3 Robot , 2019, 2019 23rd International Conference on Mechatronics Technology (ICMT).

[70]  Franck Multon,et al.  Validation of an ergonomic assessment method using Kinect data in real workplace conditions. , 2017, Applied ergonomics.

[71]  Steven J. Harrison,et al.  Journal of Neuroengineering and Rehabilitation Open Access Reliability and Validity of Pendulum Test Measures of Spasticity Obtained with the Polhemus Tracking System from Patients with Chronic Stroke , 2009 .

[72]  Hyunsoo Kim,et al.  Sensing Workers Gait Abnormality for Safety Hazard Identification , 2016 .

[73]  Hee Seok Chang,et al.  In-process monitoring of micro series spot welding using dual accelerometer system , 2019, Welding in the World.

[74]  Jinsheng Kang,et al.  Digital evaluation of sitting posture comfort in human-vehicle system under industry 4.0 framework , 2016 .

[75]  Hyunsoo Kim,et al.  Assessing the effects of slippery steel beam coatings to ironworkers' gait stability. , 2018, Applied ergonomics.

[76]  Kamiar Aminian,et al.  Capturing human motion using body‐fixed sensors: outdoor measurement and clinical applications , 2004, Comput. Animat. Virtual Worlds.

[77]  Amir H. Behzadan,et al.  Productivity Analysis of Construction Worker Activities Using Smartphone Sensors , 2016 .

[78]  Rajesh Menon,et al.  Computational imaging enables a "see-through" lens-less camera. , 2018, Optics express.

[79]  Hyunsoo Kim,et al.  Validating ambulatory gait assessment technique for hazard sensing in construction environments , 2019, Automation in Construction.

[80]  Andrew R Kemper,et al.  Occupant kinematics of the Hybrid III, THOR-M, and postmortem human surrogates under various restraint conditions in full-scale frontal sled tests , 2018, Traffic injury prevention.

[81]  Salvatore Sessa,et al.  Towards Miniaturization of a MEMS-Based Wearable Motion Capture System , 2011, IEEE Transactions on Industrial Electronics.

[82]  Dong-Eun Lee,et al.  Artificial Neural Network–Based Slip-Trip Classifier Using Smart Sensor for Construction Workplace , 2016 .

[83]  Yunyi Jia,et al.  Controlling Object Hand-Over in Human–Robot Collaboration Via Natural Wearable Sensing , 2019, IEEE Transactions on Human-Machine Systems.