Motion recognition by combining HMM and reinforcement learning

It is difficult to give a robot all possible motions beforehand in a certain environment. Therefore, the robot needs to learn how to recognize other motions and to generate its own motions autonomously for working well. These learning algorithms need an efficient way to make recognition and generation of motions work together, because they take many computing resources. This paper focuses on a generation-based recognition. Our system consists of recognition and generation modules. The fanner and latter are constructed from left-to-right hidden Markov models (HMM) and reinforcement learning (RL), respectively. When a HMM in recognition module does not work enough, the model parameters of HMM are re-estimated by using a state-value function of RL in generation module. The proposed method enables us to improve the reliability of the HMM.

[1]  Jun Morimoto,et al.  Acquisition of stand-up behavior by a real robot using hierarchical reinforcement learning , 2000, Robotics Auton. Syst..

[2]  Kenji Doya,et al.  Symbolization and Imitation Learning of Motion Sequence Using Competitive Modules , 2006 .

[3]  Yoji Yamada,et al.  An HMM-based temporal difference learning with model-updating capability for visual tracking of human communicational behaviors , 2002, Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition.

[4]  Katsunari Shibata,et al.  Gauss-sigmoid neural network , 1999, IJCNN'99. International Joint Conference on Neural Networks. Proceedings (Cat. No.99CH36339).

[5]  Shin Ishii,et al.  On-line EM Algorithm for the Normalized Gaussian Network , 2000, Neural Computation.

[6]  Kazuhito Yokoi,et al.  Generating whole body motions for a biped humanoid robot from captured human dances , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[7]  Minoru Asada,et al.  Skill acquisition and self-improvement for environmental change adaptation of mobile robot , 1998 .

[8]  Yoshihiko Nakamura,et al.  Imitation and primitive symbol acquisition of humanoids by the integrated mimesis loop , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[9]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .