On the dynamics of robot exploration learning

In this paper, the processes of exploration and of incremental learning in the robot navigation task are studied using the dynamical systems approach. A neural network model which performs the forward modeling, planning, consolidation learning and novelty rewarding is used for the robot experiments. Our experiments showed that the robot repeated a few variations of travel patterns in the beginning of the exploration, and later the robot explored more diversely in the workspace by combining and mutating the previously experienced patterns. Our analysis indicates that internal confusion due to immature learning plays the role of a catalyst in generating diverse action sequences. It is found that these diverse exploratory travels enable the robot to acquire adequate modeling of the environment in the end.

[1]  John H. Holland,et al.  COGNITIVE SYSTEMS BASED ON ADAPTIVE ALGORITHMS1 , 1978 .

[2]  J A Kelso,et al.  Dynamic pattern generation in behavioral and neural systems. , 1988, Science.

[3]  Robert F. Port,et al.  MIND IN MOTION: , 2019, Dune.

[4]  Randall D. Beer,et al.  Spatial learning for navigation in dynamic environments , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[5]  Richard S. Sutton,et al.  Neural networks for control , 1990 .

[6]  Jun Tani,et al.  An Interpretation of the "Self" From the Dynamical Systems Perspective: A Constructivist Approach , 1998 .

[7]  Richard S. Sutton,et al.  A Menu of Designs for Reinforcement Learning Over Time , 1995 .

[8]  Makoto Taiji,et al.  Dynamics of internal models in game players , 1998, adap-org/9807004.

[9]  Sebastian Thrun,et al.  Active Exploration in Dynamic Environments , 1991, NIPS.

[10]  Benjamin Kuipers,et al.  A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations , 1991, Robotics Auton. Syst..

[11]  Geoffrey E. Hinton,et al.  Learning internal representations by error propagation , 1986 .

[12]  Randall D. Beer,et al.  A Dynamical Systems Perspective on Agent-Environment Interaction , 1995, Artif. Intell..

[13]  Jun Tani.  Essential Dynamical Structure in Learnable Autonomous Robots , 1995, ECAL.

[14]  T. Gelder,et al.  It's about time: an overview of the dynamical approach to cognition , 1996 .

[15]  Linda B. Smith,et al.  A Dynamic Systems Approach to the Development of Cognition and Action , 2007, Journal of Cognitive Neuroscience.

[16]  John H. Holland,et al.  Cognitive systems based on adaptive algorithms , 1977, SGAR.

[17]  Donald A. Waterman,et al.  Pattern-Directed Inference Systems , 1981, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Jordan B. Pollack,et al.  The induction of dynamical recognizers , 1991, Machine Learning.

[19]  Maja J. Mataric,et al.  Integration of representation into goal-driven behavior-based robots , 1992, IEEE Trans. Robotics Autom..

[20]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[21]  James L. McClelland,et al.  Parallel distributed processing: explorations in the microstructure of cognition, vol. 1: foundations , 1986 .

[22]  Jean-Arcady Meyer,et al.  From Animals to Animats: Proceedings of The First International Conference on Simulation of Adaptive Behavior (Complex Adaptive Systems) , 1990 .

[23]  Jun Tani,et al.  Model-based learning for mobile robot navigation from the dynamical systems perspective , 1996, IEEE Trans. Syst. Man Cybern. Part B.

[24]  Jürgen Schmidhuber,et al.  A possibility for implementing curiosity and boredom in model-building neural controllers , 1991 .

[25]  M. Kawato,et al.  A hierarchical neural-network model for control and learning of voluntary movement , 2004, Biological Cybernetics.