Collection and Analysis of Human Upper Limbs Motion Features for Collaborative Robotic Applications

(1) Background: The technologies of Industry 4.0 are increasingly promoting an operation of human motion prediction for improvement of the collaboration between workers and robots. The purposes of this study were to fuse the spatial and inertial data of human upper limbs for typical industrial pick and place movements and to analyze the collected features from the future perspective of collaborative robotic applications and human motion prediction algorithms. (2) Methods: Inertial Measurement Units and a stereophotogrammetric system were adopted to track the upper body motion of 10 healthy young subjects performing pick and place operations at three different heights. From the obtained database, 10 features were selected and used to distinguish among pick and place gestures at different heights. Classification performances were evaluated by estimating confusion matrices and F1-scores. (3) Results: Values on matrices diagonals were definitely greater than those in other positions. Furthermore, F1-scores were very high in most cases. (4) Conclusions: Upper arm longitudinal acceleration and markers coordinates of wrists and elbows could be considered representative features of pick and place gestures at different heights, and they are consequently suitable for the definition of a human motion prediction algorithm to be adopted in effective collaborative robotics industrial applications.

[1]  Nima Enayati,et al.  A Neural Network-Based Approach for Trajectory Planning in Robot–Human Handover Tasks , 2016, Front. Robot. AI.

[2]  Stefano Pastorelli,et al.  Evaluation of functional methods for human movement modelling. , 2015, Acta of bioengineering and biomechanics.

[3]  Marcin Basiaga,et al.  Technological capabilities of surface layers formation on implant made of Ti-6Al-4V ELI alloy. , 2015, Acta of bioengineering and biomechanics.

[4]  Ryad Chellali,et al.  Predicting Arm Movements A Multi-Variate LSTM Based Approach for Human-Robot Hand Clapping Games , 2018, 2018 27th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[5]  Jessica K. Hodgins,et al.  Guide to the Carnegie Mellon University Multimodal Activity (CMU-MMAC) Database , 2008 .

[6]  Cristian Sminchisescu,et al.  Human3.6M: Large Scale Datasets and Predictive Methods for 3D Human Sensing in Natural Environments , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Liva Ralaivola,et al.  Confusion Matrix Stability Bounds for Multiclass Classification , 2012, ArXiv.

[8]  Stefano Pastorelli,et al.  Influence of Human Limb Motion Speed in a Collaborative Hand-over Task , 2018, ICINCO.

[9]  Andrea Maria Zanchettin,et al.  Optimal Proactive Path Planning for Collaborative Robots in Industrial Contexts , 2019, 2019 International Conference on Robotics and Automation (ICRA).

[10]  Martin Buss,et al.  Human-Robot Collaboration: a Survey , 2008, Int. J. Humanoid Robotics.

[11]  Terrence Fong,et al.  A Survey of Methods for Safe Human-Robot Interaction , 2017, Found. Trends Robotics.

[12]  Taku Komura,et al.  A Deep Learning Framework for Character Motion Synthesis and Editing , 2016, ACM Trans. Graph..

[13]  Stefano Pastorelli,et al.  Collision Avoidance Algorithm for Collaborative Robotics , 2017, Int. J. Autom. Technol..

[14]  Kazuhiro Kosuge,et al.  Progress and prospects of the human–robot collaboration , 2017, Autonomous Robots.

[15]  Danica Kragic,et al.  Anticipating Many Futures: Online Human Motion Prediction and Generation for Human-Robot Interaction , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Laura Gastaldi,et al.  Comparison of Different Motion Capture Setups for Gait Analysis : Validation of spatio-temporal parameters estimation , 2018, 2018 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[17]  Julie A. Shah,et al.  Fast target prediction of human reaching motion for cooperative human-robot manipulation tasks using time series classification , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Anita Bagley,et al.  A method for determination of upper extremity kinematics. , 2002, Gait & posture.

[19]  Lin Sun,et al.  Activity Recognition on an Accelerometer Embedded Mobile Phone with Varying Positions and Orientations , 2010, UIC.

[20]  Harald Aschemann,et al.  Safe and Efficient Human-Robot Collaboration Part I: Estimation of Human Arm Motions , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[21]  Stefano Paolo Pastorelli,et al.  Upper Limbs Motion Tracking for Collaborative Robotic Applications , 2020 .

[22]  Anne Moore,et al.  The effects of cycle time on the physical demands of a repetitive pick-and-place task. , 2007, Applied ergonomics.

[23]  Dmitry Berenson,et al.  Human-robot collaborative manipulation planning using early prediction of human motion , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Tido Röder,et al.  Documentation Mocap Database HDM05 , 2007 .

[25]  Wenlong Zhang,et al.  Collision-free trajectory planning in human-robot interaction through hand movement prediction from vision , 2017, 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids).

[26]  Ibrahim I. Esat,et al.  Articular human joint modelling , 2009, Robotica.

[27]  Stefano Pastorelli,et al.  Tilt-Twist Method Using Inertial Sensors to Assess Spinal Posture During Gait , 2019, RAAD.

[28]  Ruzena Bajcsy,et al.  Berkeley MHAD: A comprehensive Multimodal Human Action Database , 2013, 2013 IEEE Workshop on Applications of Computer Vision (WACV).

[29]  Matthias Althoff,et al.  Overapproximative arm occupancy prediction for human-robot co-existence built from archetypal movements , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[30]  Jessica K. Hodgins,et al.  Realtime style transfer for unlabeled heterogeneous human motion , 2015, ACM Trans. Graph..

[31]  M. de Looze,et al.  Effect of horizontal pick and place locations on shoulder kinematics , 2015, Ergonomics.

[32]  Tullio Tolio,et al.  A probabilistic approach to workspace sharing for human–robot cooperation in assembly tasks , 2016 .

[33]  Otmar Hilliges,et al.  Learning Human Motion Models for Long-Term Predictions , 2017, 2017 International Conference on 3D Vision (3DV).

[34]  Sotiris Makris,et al.  Human–robot interaction review and challenges on task planning and programming , 2016, Int. J. Comput. Integr. Manuf..