Learning of sensorimotor behaviors by a SASE agent for vision-based navigation

In this paper, we propose a model to develop robotspsila covert and overt behaviors by using reinforcement and supervised learning jointly. The covert behaviors are handled by a motivational system, which is achieved through reinforcement learning. The overt behaviors are directly selected by imposing supervised signals. Instead of dealing with problems in controlled environments with a low-dimensional state space, our model is applied for the learning in non-stationary environments. Locally balanced incremental hierarchical discriminant regression (LBIHDR) tree is introduce to be the engine of cognitive mapping. Its balanced coarse-to-fine tree structure guarantees real-time retrieval in self-generated high-dimensional state space. Furthermore, K-nearest neighbor strategy is adopted to reduce training time complexity. Vision-based outdoor navigation are used as challenging task examples. In the experiment, the mean square error of heading direction is 0deg for re-substitution test and 1.1269deg for disjoint test, which allows the robot to drive without a big deviation from the correct path we expected. Compared with IHDR (W.S. Hwang and J. Weng, 2007), LBIHDR reduced the mean square error by 0.252deg and 0.5052deg, using re-substitution and disjoint test, respectively.

[1]  Dean Pomerleau,et al.  ALVINN, an autonomous land vehicle in a neural network , 2015 .

[2]  Juyang Weng,et al.  Incremental Hierarchical Discriminant Regression , 2007, IEEE Transactions on Neural Networks.

[3]  Philippe Gaussier,et al.  Visual navigation in an open environment without map , 1997, Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems. Innovative Robotics for Real-World Applications. IROS '97.

[4]  Juyang Weng,et al.  A theory for mentally developing robots , 2002, Proceedings 2nd International Conference on Development and Learning. ICDL 2002.

[5]  Juyang Weng,et al.  Hierarchical Discriminant Regression , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Christian Balkenius,et al.  Attention as selection-for-action: a scheme for active perception , 1999, 1999 Third European Workshop on Advanced Mobile Robots (Eurobot'99). Proceedings (Cat. No.99EX355).

[7]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[8]  Ian Horswill,et al.  Polly: A Vision-Based Artificial Agent , 1993, AAAI.

[9]  James L. McClelland,et al.  Autonomous Mental Development by Robots and Animals , 2001, Science.

[10]  Todd M. Jochem.  USING VIRTUAL ACTIVE VISION TOOLS TO IMPROVE AUTONOMOUS DRIVING TASKS , 1994 .

[11]  Dana H. Ballard,et al.  Active Perception and Reinforcement Learning , 1990, Neural Computation.

[12]  Philippe Gaussier,et al.  Learning to build visual categories from perception-action associations , 1997, Proceedings of the 1997 IEEE/RSJ International Conference on Intelligent Robot and Systems. Innovative Robotics for Real-World Applications. IROS '97.

[13]  Charles E. Thorpe,et al.  Vision-based neural network road and intersection detection and traversal , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[14]  John Hallam,et al.  IEEE International Joint Conference on Neural Networks , 2005 .

[15]  Masayuki Inaba,et al.  Visual navigation using view-sequenced route representation , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[16]  Sridhar Mahadevan,et al.  A reinforcement learning model of selective visual attention , 2001, AGENTS '01.