Utilizing unsupervised weightless neural network as autonomous states classifier in reinforcement learning algorithm

An implementation of reinforcement learning algorithm in an autonomous system requires knowledge expert to specify anticipated states, actions and rewards; and the algorithm will autonomously discover a near optimal behaviour for the system through trial-and-error interactions with its environment. The information on anticipated states are usually extracted from data streams and pre-programmed based on the knowledge expert interpretation of the data thus making the reinforcement learning algorithm rigid to only handles anticipated circumstances and the system will not be able to optimize. As an alternative, in this paper we explore the use of AUTOWiSARD, an unsupervised weightless neural network which will autonomously classify the states based on sensor information and then used by Q-learning, a reinforcement learning algorithm in order find near optimal behavior. The implementation will be demonstrated in an autonomous mobile robot simulation and the outcome will be presented and discussed.

[1]  Chen Xia,et al.  A Reinforcement Learning Method of Obstacle Avoidance for Industrial Mobile Vehicles in Unknown Environments Using Neural Network , 2015, IEEM 2015.

[2]  Felipe Maia Galvão França,et al.  AUTOWISARD: Unsupervised Modes for the WISARD , 2001, IWANN.

[3]  Ferat Sahin,et al.  Autonomous navigation based on a Q-learning algorithm for a robot in a real environment , 2008, 2008 IEEE International Conference on System of Systems Engineering.

[4]  Felipe Maia Galvão França,et al.  Multilingual part-of-speech tagging with weightless neural networks , 2015, Neural Networks.

[5]  Nadia Nedjah,et al.  Weightless neural systems , 2016, Neurocomputing.

[6]  Jonathan K. Alt.  Learning from Noisy and Delayed Rewards: The Value of Reinforcement Learning to Defense Modeling and Simulation , 2012 .

[7]  Felipe M. G. França,et al.  Validating an unsupervised weightless perceptron , 2002, Proceedings of the 9th International Conference on Neural Information Processing, 2002. ICONIP '02..

[8]  Yusman Yusof,et al.  Simulation of mobile robot navigation utilizing reinforcement and unsupervised weightless neural network learning algorithm , 2015, 2015 IEEE Student Conference on Research and Development (SCOReD).

[9]  Klaus Obermayer,et al.  Autonomous Learning of State Representations for Control: An Emerging Field Aims to Autonomously Learn State Representations for Reinforcement Learning Agents from Their Real-World Sensor Observations , 2015, KI - Künstliche Intelligenz.

[10]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[11]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.