Robotics and Multi-agent Systems Robustness in the Long Run: Auto-teaching vs Anticipation in Evolutionary Robotics

In Evolutionary Robotics, auto-teaching networks, neural networks that modify their own weights during the life-time of the robot, have been shown to be powerful architectures to develop adaptive controllers. Unfortunately, when run for a longer period of time than that used during evolution, the long-term behavior of such networks can become unpredictable. This paper gives an example of such dangerous behavior, and proposes an alternative solution based on anticipation: as in auto-teaching networks, a secondary network is evolved, but its outputs try to predict the next state of the robot sensors. The weights of the action network are adjusted using some back-propagation procedure based on the errors made by the anticipatory network. First results – in simulated environments – show a tremendous increase in robustness of the long-term behavior of the controller.

[1]  Stefano Nolfi.  How Learning and Evolution Interact: The Case of a Learning Task which Differs from the Evolutionary Task , 1999, Adapt. Behav..

[2]  Stefano Nolfi,et al.  Good teaching inputs do not correspond to desired responses in ecological neural networks , 1994, Neural Processing Letters.

[3]  Stefano Nolfi,et al.  How Co-Evolution can Enhance the Adaptive Power of Artificial Evolution: Implications for Evolutionary Robotics , 1998, EvoRobot.

[4]  Inman Harvey,et al.  Is There Another New Factor in Evolution? , 1996, Evolutionary Computation.

[5]  Naftali Tishby,et al.  The information bottleneck method , 2000, ArXiv.

[6]  Stefano Nolfi,et al.  Learning to Adapt to Changing Environments in Evolving Neural Networks , 1996, Adapt. Behav..

[7]  Jeffrey L. Elman,et al.  Learning and Evolution in Neural Networks , 1994, Adapt. Behav..

[8]  Stefano Nolfi,et al.  Desired answers do not correspond to good teaching inputs in ecological neural networks , 2000 .

[9]  Peter Stone,et al.  Anticipation as a key for collaboration in a team of agents: a case study in robotic soccer , 1999, Optics East.

[10]  A. Noë,et al.  A sensorimotor account of vision and visual consciousness. , 2001, The Behavioral and brain sciences.

[11]  Ronald C. Arkin,et al.  Anticipatory robot navigation by simultaneously localizing and building a cognitive map , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[12]  Wolfgang Stolzmann,et al.  An Introduction to Anticipatory Classifier Systems , 1999, Learning Classifier Systems.

[13]  Jean-Gabriel Ganascia,et al.  Learning Strategies in Games by Anticipation , 1997, IJCAI.

[14]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[15]  Yannis Dimopoulos,et al.  Use of some sensitivity criteria for choosing networks with good generalization ability , 1995, Neural Processing Letters.