Human motion prediction for human-robot collaboration

Abstract In human-robot collaborative manufacturing, industrial robots would work alongside human workers who jointly perform the assigned tasks seamlessly. A human-robot collaborative manufacturing system is more customised and flexible than conventional manufacturing systems. In the area of assembly, a practical human-robot collaborative assembly system should be able to predict a human worker’s intention and assist human during assembly operations. In response to the requirement, this research proposes a new human-robot collaborative system design. The primary focus of the paper is to model product assembly tasks as a sequence of human motions. Existing human motion recognition techniques are applied to recognise the human motions. Hidden Markov model is used in the motion sequence to generate a motion transition probability matrix. Based on the result, human motion prediction becomes possible. The predicted human motions are evaluated and applied in task-level human-robot collaborative assembly.

[1]  S. Abdul-Kareem,et al.  RETRACTED ARTICLE: Static hand gesture recognition using neural networks , 2014, Artificial Intelligence Review.

[2]  Aaron F. Bobick,et al.  Probabilistic human action prediction and wait-sensitive planning for responsive human-robot collaboration , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[3]  Rainer Müller,et al.  Robot Workmate: A Trustworthy Coworker for the Continuous Automotive Assembly Line and its Implementation☆ , 2016 .

[4]  Yun Fu,et al.  Prediction of Human Activity by Discovering Temporal Sequence Patterns , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Michael E. Tipping.  Sparse Bayesian Learning and the Relevance Vector Machine , 2001, J. Mach. Learn. Res..

[6]  Rini Akmeliawati,et al.  Hidden Markov model for human to computer interaction: a study on human hand gesture recognition , 2011, Artificial Intelligence Review.

[7]  Lihui Wang,et al.  Vision-guided active collision avoidance for human-robot collaborations , 2013 .

[8]  Sotiris Manitsaris,et al.  New Challenges for Human-Robot Collaboration in an Industrial Context: Acceptability and Natural Collaboration , 2016, RO-MAN 2016.

[9]  Shuzhi Sam Ge,et al.  Human–Robot Collaboration Based on Motion Intention Estimation , 2014, IEEE/ASME Transactions on Mechatronics.

[10]  Fang Yuan,et al.  Static hand gesture recognition based on HOG characters and support vector machines , 2013, 2013 2nd International Symposium on Instrumentation and Measurement, Sensor Network and Automation (IMSNA).

[11]  Roland Menassa,et al.  Levels of human and robot collaboration for automotive manufacturing , 2012, PerMIS.

[12]  Kirsten E. Kaplan,et al.  Improving inclusion segmentation task performance through human-intent based human-robot collaboration , 2016, 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[13]  Michael S. Ryoo,et al.  Human activity prediction: Early recognition of ongoing activities from streaming videos , 2011, 2011 International Conference on Computer Vision.

[14]  Jin Zhang,et al.  Learning hierarchical spatio-temporal pattern for human activity prediction , 2016, J. Vis. Commun. Image Represent..

[15]  James Patton,et al.  Reaching is Better When You Get What You Want: Realtime Feedback of Intended Reaching Trajectory Despite an Unstable Environment , 2016, Front. Behav. Neurosci..

[16]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[17]  Przemyslaw A. Lasota,et al.  Toward safe close-proximity human-robot interaction with standard industrial robots , 2014, 2014 IEEE International Conference on Automation Science and Engineering (CASE).

[18]  Dmitry Berenson,et al.  Human-robot collaborative manipulation planning using early prediction of human motion , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[19]  Lihui Wang,et al.  Review: Advances in 3D data acquisition and processing for industrial applications , 2010 .

[20]  Dmitry Berenson,et al.  Goal Set Inverse Optimal Control and Iterative Replanning for Predicting Human Reaching Motions in Shared Workspaces , 2016, IEEE Transactions on Robotics.

[21]  Giovanni Mirabella,et al.  Should I stay or should I go? Conceptual underpinnings of goal-directed actions , 2014, Front. Syst. Neurosci..

[22]  Darwin G. Caldwell,et al.  Learning Controllers for Reactive and Proactive Behaviors in Human–Robot Collaboration , 2016, Front. Robot. AI.

[23]  Paolo Rocco,et al.  Towards safe human-robot interaction in robotic cells: An approach based on visual tracking and intention estimation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Alexander Verl,et al.  Cooperation of human and machines in assembly lines , 2009 .

[25]  J V Tu,et al.  Advantages and disadvantages of using artificial neural networks versus logistic regression for predicting medical outcomes. , 1996, Journal of clinical epidemiology.

[26]  James Patton,et al.  I Meant to Do That: Determining the Intentions of Action in the Face of Disturbances , 2015, PloS one.

[27]  Lihui Wang,et al.  Depth camera based collision avoidance via active robot control , 2014 .

[28]  Lihui Wang,et al.  Gesture recognition for human-robot collaboration: A review , 2017, International Journal of Industrial Ergonomics.

[29]  Hao Ding,et al.  Collaborative behavior design of industrial robots for multiple human-robot collaboration , 2013, IEEE ISR 2013.