Analysis and Feedback of Movement in Manual Assembly Process

A manual assembly has an important role in the manufacturing of products with small lot sizes and high variation. Becoming skilled manual labor requires knowledge transfers offered by human experts through a training process. To reduce the dependency of human experts, this paper introduces a framework called “Virtual Trainer” that incorporates the current state of the art marker-less RGB human pose estimation, activity detection for assembly step recognition, and training feedback through a multi-media presentation includes score evaluation and semantic description of trainee performance. Furthermore, the detailed transcript of each step and 3-D visualization compares to ideal movements also presented. The experimental design for evaluating the effectiveness and hypothesis is given.

[1]  Nasser Kehtarnavaz,et al.  Deep Learning-based Human Pose Estimation: A Survey , 2020, ACM Comput. Surv..

[2]  H Ghasemzadeh,et al.  Coordination Analysis of Human Movements With Body Sensor Networks: A Signal Processing Model to Evaluate Baseball Swings , 2011, IEEE Sensors Journal.

[3]  Ioannis A. Kakadiaris,et al.  3D Human pose estimation: A review of the literature and analysis of covariates , 2016, Comput. Vis. Image Underst..

[4]  Markus Funk,et al.  Augmented reality at the workplace : a context-aware assistive system using in-situ projection , 2016 .

[5]  R. Riener,et al.  Augmented visual, auditory, haptic, and multimodal feedback in motor learning: A review , 2012, Psychonomic Bulletin & Review.

[6]  Antonio Barrientos,et al.  A training system for Industry 4.0 operators in complex assemblies based on virtual reality and process mining , 2019, Robotics Comput. Integr. Manuf..

[7]  Franco Tecchia,et al.  Evaluating virtual reality and augmented reality training for industrial maintenance and assembly tasks , 2015, Interact. Learn. Environ..

[8]  Norbert Link,et al.  Worker Behavior Interpretation for Flexible Production , 2009 .

[9]  Yaser Sheikh,et al.  OpenPose: Realtime Multi-Person 2D Pose Estimation Using Part Affinity Fields , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Timo Engelke,et al.  An augmented reality training platform for assembly and maintenance skills , 2013, Robotics Auton. Syst..

[11]  Petros Daras,et al.  Motion analysis: Action detection, recognition and evaluation based on motion capture data , 2018, Pattern Recognit..

[12]  A KakadiarisIoannis,et al.  3D Human pose estimation , 2016 .

[13]  Taku Komura,et al.  A Virtual Reality Dance Training System Using Motion Capture Technology , 2011, IEEE Transactions on Learning Technologies.