Simple and complex behavior learning using behavior hidden Markov model and CobART

This paper proposes behavior learning and generation models for simple and complex behaviors of robots using unsupervised learning methods. While the simple behaviors are modeled by simple-behavior learning model (SBLM), complex behaviors are modeled by complex-behavior learning model (CBLM) which uses previously learned simple or complex behaviors. Both models include behavior categorization, behavior modeling, and behavior generation phases. In the behavior categorization phase, sensory data are categorized using correlation based adaptive resonance theory (CobART) network that generates motion primitives corresponding to robot's base abilities. In the behavior modeling phase, a modified version of hidden Markov model (HMM), is called Behavior-HMM, is used to model the relationships among the motion primitives in a finite state stochastic network. At the same time, a motion generator which is an artificial neural network (ANN) is trained for each motion primitive to learn essential robot motor commands. In the behavior generation phase, a motion primitive sequence that can perform the desired task is generated according to the previously learned Behavior-HMMs at the higher level. Then, in the lower level, these motion primitives are executed by the motion generator which is specifically trained for the corresponding motion primitive. The transitions between the motion primitives are done according to observed sensory data and probabilistic weights assigned to each transition during the learning phase. The proposed models are not constructed for one specific behavior, but are intended to be bases for all behaviors. The behavior learning capabilities of the model is extended by integrating previously learned behaviors hierarchically which is referred as CBLM. Hence, new behaviors can take advantage of already discovered behaviors. Performed experiments on a robot simulator show that simple and complex-behavior learning models can generate requested behaviors effectively.

[1]  Sridhar Mahadevan,et al.  Learning hierarchical models of activity , 2004, 2004 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (IEEE Cat. No.04CH37566).

[2]  Jun Tani,et al.  On the Dynamics of Robot Exploration Learning , 1999, ECAL.

[3]  Jun Tani,et al.  How Hierarchical Control Self-organizes in Artificial Adaptive Systems , 2005, Adapt. Behav..

[4]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[5]  Majid Nili Ahmadabadi,et al.  Conceptual Imitation Learning: An Application to Human-robot Interaction , 2010, ACML.

[6]  Malik Ghallab,et al.  Learning the behavior model of a robot , 2011, Auton. Robots.

[7]  Stephen Grossberg,et al.  ART 2-A: An adaptive resonance algorithm for rapid category learning and recognition , 1991, Neural Networks.

[8]  Jochen J. Steil,et al.  Reaching movement generation with a recurrent neural network based on learning inverse kinematics for the humanoid robot iCub , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[9]  S. Grossberg,et al.  ART 2: self-organization of stable category recognition codes for analog input patterns. , 1987, Applied optics.

[10]  I. Chen,et al.  Planning algorithms for s-curve trajectories , 2007, 2007 IEEE/ASME international conference on advanced intelligent mechatronics.

[11]  Ronald C. Arkin,et al.  An Behavior-based Robotics , 1998 .

[12]  Jun Tani,et al.  Codevelopmental Learning Between Human and Humanoid Robot Using a Dynamic Neural-Network Model , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[13]  F. Alpaslan,et al.  CobART: Correlation Based Adaptive Resonance Theory , 2009, 2009 17th Mediterranean Conference on Control and Automation.

[14]  Simon Haykin,et al.  Neural Networks and Learning Machines , 2010 .

[15]  Shohei Kato,et al.  Imitative motion generation for humanoid robots based on the motion knowledge learning and reuse , 2009, 2009 IEEE International Conference on Systems, Man and Cybernetics.

[16]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[17]  Alejandro Dizan Vasquez Govea.  Growing Hidden Markov Models , 2010 .

[18]  Maja J. Mataric,et al.  Demonstration-Based Behavior and Task Learning , 2006, AAAI Spring Symposium: To Boldly Go Where No Human-Robot Team Has Gone Before.

[19]  Monica N. Nicolescu,et al.  Understanding human intentions via Hidden Markov Models in autonomous mobile robots , 2008, 2008 3rd ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[20]  Edsger W. Dijkstra,et al.  A note on two problems in connexion with graphs , 1959, Numerische Mathematik.

[21]  Jun Tani,et al.  Learning to Generate Combinatorial Action Sequences Utilizing the Initial Sensitivity of Deterministic Dynamical Systems , 2003, IWANN.

[22]  Toyoaki Nishida,et al.  Incremental clustering of gesture patterns based on a self organizing incremental neural network , 2009, 2009 International Joint Conference on Neural Networks.

[23]  Naoyuki Kubota,et al.  Computational intelligence for structured learning of a partner robot based on imitation , 2005, Inf. Sci..

[24]  Rodney A. Brooks,et al.  A Robust Layered Control Syste For A Mobile Robot , 2022 .

[25]  Malik Ghallab,et al.  Robot introspection through learned hidden Markov models , 2006, Artif. Intell..

[26]  Manuela Veloso,et al.  Automated Robot Behavior Recognition Applied to Robotic Soccer , 1999 .

[27]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[28]  Malik Ghallab,et al.  Learning how to combine sensory-motor functions into a robust behavior , 2008, Artif. Intell..

[29]  Stephen Grossberg,et al.  A massively parallel architecture for a self-organizing neural pattern recognition machine , 1988, Comput. Vis. Graph. Image Process..

[30]  Jeffrey L. Krichmar,et al.  Evolutionary robotics: The biology, intelligence, and technology of self-organizing machines , 2001, Complex..

[31]  Andrew J. Viterbi,et al.  Error bounds for convolutional codes and an asymptotically optimum decoding algorithm , 1967, IEEE Trans. Inf. Theory.

[32]  Jun Tani,et al.  Learning to generate articulated behavior through the bottom-up and the top-down interaction processes , 2003, Neural Networks.

[33]  Robin R. Murphy,et al.  Introduction to AI Robotics , 2000 .

[34]  Stephen Grossberg,et al.  Fuzzy ARTMAP: A neural network architecture for incremental supervised learning of analog multidimensional maps , 1992, IEEE Trans. Neural Networks.

[35]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[36]  Stephen Grossberg,et al.  Fuzzy ART: Fast stable learning and categorization of analog patterns by an adaptive resonance system , 1991, Neural Networks.

[37]  Dana Kulic,et al.  Incremental learning of human behaviors using hierarchical hidden Markov models , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[38]  Ferda Nur Alpaslan,et al.  Behavior categorization using Correlation Based Adaptive Resonance Theory , 2009, 2009 17th Mediterranean Conference on Control and Automation.

[39]  David M. Skapura,et al.  Neural networks - algorithms, applications, and programming techniques , 1991, Computation and neural systems series.

[40]  Christian Laugier,et al.  Growing Hidden Markov Models: An Incremental Tool for Learning and Predicting Human and Vehicle Motion , 2009, Int. J. Robotics Res..

[41]  Stephen Grossberg,et al.  ARTMAP: supervised real-time learning and classification of nonstationary data by a self-organizing neural network , 1991, [1991 Proceedings] IEEE Conference on Neural Networks for Ocean Engineering.

[42]  Eytan Ruppin,et al.  The evolution of imitation and mirror neurons in adaptive agents , 2005, Cognitive Systems Research.

[43]  Dana Kulic,et al.  Incremental Learning, Clustering and Hierarchy Formation of Whole Body Motion Patterns using Adaptive Hidden Markov Chains , 2008, Int. J. Robotics Res..

[44]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .