Human behavior learning for robot in joint space

Many robots, such as human guide robots and service robots, need to learn the human behavior such that the robots can merge into human society. This learning process is in task space. However, the desired trajectories should be in joint space such that each joint can be moved like human. This needs inverse kinematics. In most of cases, the inverse kinematics do not have analytical solutions. Few training methods work in joint space directly, because they need dynamic time warping to remove speed influence. Both inverse kinematics and dynamic time warping destroy the accuracy of the learning.In this paper, the desired trajectory is trained in joint space without the dynamic time warping. In order to learn the demonstrations in joint space, we use two techniques, Lloyd?s algorithm and modified hidden Markov model, to solve the problems in joint space learning. Since the desired trajectories are the joint angles, they can be applied directly without inverse kinematics. Experimental results show that the proposed algorithm works well for human behavior learning.

[1]  IjspeertAuke Jan,et al.  Dynamical movement primitives , 2013 .

[2]  Blake Hannaford,et al.  Raven-II: An Open Platform for Surgical Robotics Research , 2013, IEEE Transactions on Biomedical Engineering.

[3]  Ales Ude,et al.  Trajectory generation from noisy positions of object features for teaching robot paths , 1993, Robotics Auton. Syst..

[4]  Bernhard Schölkopf,et al.  Probabilistic movement modeling for intention inference in human–robot interaction , 2013, Int. J. Robotics Res..

[5]  Andrew J. Viterbi,et al.  Error bounds for convolutional codes and an asymptotically optimum decoding algorithm , 1967, IEEE Trans. Inf. Theory.

[6]  Eamonn J. Keogh,et al.  Experimental comparison of representation methods and distance measures for time series data , 2010, Data Mining and Knowledge Discovery.

[7]  Mark W. Spong,et al.  Robot dynamics and control , 1989 .

[8]  P. Gader,et al.  Generalized Hidden Markov Models — Part I : Theoretical Frameworks , 2008 .

[9]  Archan Misra,et al.  TODMIS: mining communities from trajectories , 2013, CIKM.

[10]  Stefan Schaal,et al.  Learning to Control in Operational Space , 2008, Int. J. Robotics Res..

[11]  L. Baum,et al.  A Maximization Technique Occurring in the Statistical Analysis of Probabilistic Functions of Markov Chains , 1970 .

[12]  Aude Billard,et al.  Dynamical System Modulation for Robot Learning via Kinesthetic Demonstrations , 2008, IEEE Transactions on Robotics.

[13]  Ka Keung Lee,et al.  Human behavior learning and transfer , 2005 .

[14]  Katsushi Ikeuchi,et al.  Extraction of essential interactions through multiple observations of human demonstrations , 2003, IEEE Trans. Ind. Electron..

[15]  Paul D. Gader,et al.  Generalized hidden Markov models. I. Theoretical frameworks , 2000, IEEE Trans. Fuzzy Syst..

[16]  Robert M. Gray,et al.  An Algorithm for Vector Quantizer Design , 1980, IEEE Trans. Commun..

[17]  Ramayya Krishnan,et al.  Understanding Sequential Decisions via Inverse Reinforcement Learning , 2013, 2013 IEEE 14th International Conference on Mobile Data Management.

[18]  Jun Nakanishi,et al.  Dynamical Movement Primitives: Learning Attractor Models for Motor Behaviors , 2013, Neural Computation.

[19]  S. P. Lloyd,et al.  Least squares quantization in PCM , 1982, IEEE Trans. Inf. Theory.

[20]  Aude Billard,et al.  Learning Non-linear Multivariate Dynamics of Motion in Robotic Manipulators , 2011, Int. J. Robotics Res..

[21]  Gernot A. Fink,et al.  Markov Models for Pattern Recognition: From Theory to Applications , 2007 .

[22]  Siyuan Liu,et al.  Robust Bayesian Inverse Reinforcement Learning with Sparse Behavior Noise , 2014, AAAI.

[23]  Mykola Pechenizkiy,et al.  HyDR-MI: A hybrid algorithm to reduce dimensionality in multiple instance learning , 2013, Inf. Sci..

[24]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[25]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[26]  Jun Tani,et al.  Dynamic and interactive generation of object handling behaviors by a small humanoid robot using a dynamic neural network model , 2006, Neural Networks.

[27]  Andrew T. Irish,et al.  Trajectory Learning for Robot Programming by Demonstration Using Hidden Markov Model and Dynamic Time Warping , 2012, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[28]  Yangsheng Xu,et al.  Human action learning via hidden Markov model , 1997, IEEE Trans. Syst. Man Cybern. Part A.

[29]  Yasuo Kuniyoshi,et al.  Embodied basis of invariant features in execution and perception of whole-body dynamic actions - knacks and focuses of Roll-and-Rise motion , 2004, Robotics Auton. Syst..

[30]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[31]  Frank Chongwoo Park,et al.  Natural Movement Generation Using Hidden Markov Models and Principal Components , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[32]  Pieter Abbeel,et al.  Superhuman performance of surgical tasks by robots using iterative learning from human-guided demonstrations , 2010, 2010 IEEE International Conference on Robotics and Automation.