Deep learning-based human motion recognition for predictive context-aware human-robot collaboration

Abstract Timely context awareness is key to improving operation efficiency and safety in human-robot collaboration (HRC) for intelligent manufacturing. Visual observation of human workers’ motion provides informative clues about the specific tasks to be performed, thus can be explored for establishing accurate and reliable context awareness. Towards this goal, this paper investigates deep learning as a data driven technique for continuous human motion analysis and future HRC needs prediction, leading to improved robot planning and control in accomplishing a shared task. A case study in engine assembly is carried out to validate the feasibility of the proposed method.

[1]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[2]  Yun Fu,et al.  Prediction of Human Activity by Discovering Temporal Sequence Patterns , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Lihui Wang,et al.  Gesture recognition for human-robot collaboration: A review , 2017, International Journal of Industrial Ergonomics.

[4]  Xiaogang Wang,et al.  Deep Learning Face Representation from Predicting 10,000 Classes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[5]  Alois Knoll,et al.  Human activity recognition in the context of industrial human-robot interaction , 2014, Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2014 Asia-Pacific.

[6]  Alexander Verl,et al.  Cooperation of human and machines in assembly lines , 2009 .

[7]  Aaron F. Bobick,et al.  Probabilistic human action prediction and wait-sensitive planning for responsive human-robot collaboration , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[8]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[9]  Lihui Wang,et al.  Human motion prediction for human-robot collaboration , 2017 .

[10]  Gregory D. Abowd,et al.  Towards a Better Understanding of Context and Context-Awareness , 1999, HUC.

[11]  Julie A. Shah,et al.  Fast target prediction of human reaching motion for cooperative human-robot manipulation tasks using time series classification , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[13]  Ivan Laptev,et al.  Learning and Transferring Mid-level Image Representations Using Convolutional Neural Networks , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  S. Abdul-Kareem,et al.  RETRACTED ARTICLE: Static hand gesture recognition using neural networks , 2014, Artificial Intelligence Review.