Single Agent Learning Algorithms for Decision making in Diagnostic Applications

The output of the system is a sequence of actions in some applications. There is no such measure as the best action in any in-between state; an action is excellent if it is part of a good policy. A single action is not important; the policy is important that is the sequence of correct actions to reach the goal. To be able to generate a policy the machine learning programs should able to assess the quality of policies and learn from past good action sequences. Learning is the basic capacity of intelligent agents. An agent changes its behaviour based on its previous experiences through learning. An intelligent agent must be formalized by knowledge and be able to act on this knowledge. In many single-agent systems for learning the policy of an agent in uncertain environments, the reinforcement learning techniques have been applied successfully. Many existing singleagent models for sequential decision making are derived from a general model and are distinguished by assumptions. Q-learning algorithms are used for this purpose. Single agent learning model is given in this paper. Four single agent reinforcement learning algorithms are implemented and results are compared. Single agent Q-learning Algorithm and Sarsa Learning Algorithm gives some results for the problem. However adding eligibility traces in single agent learning algorithms i.e. Q(λ) learning and Sarsa(λ) learning gives performs better than the previous algorithms. The paper shows the results of all four algorithms and performance comparisons among them.

[1]  Majid Nili Ahmadabadi,et al.  A Study on Expertise of Agents and Its Effects on Cooperative $Q$-Learning , 2007, IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics).

[2]  Mohammad Ali Abbasi,et al.  Reinforcement Distribution in a Team of Cooperative Q-learning Agents , 2008, 2008 Ninth ACIS International Conference on Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing.

[3]  Costas Tsatsoulis,et al.  Learning Communication Strategies in Multiagent Systems , 1998, Applied Intelligence.

[4]  Jun Zeng,et al.  Cooperative reinforcement learning algorithm to distributed power system based on Multi-Agent , 2009, 2009 3rd International Conference on Power Electronics Systems and Applications (PESA).

[5]  Jun-Yuan Tao,et al.  Cooperative Strategy Learning in Multi-Agent Environment with Continuous State Space , 2006, 2006 International Conference on Machine Learning and Cybernetics.

[6]  Ronen I. Brafman,et al.  Learning to Coordinate Efficiently: A Model-based Approach , 2003, J. Artif. Intell. Res..

[7]  Hyo-Sung Ahn,et al.  A survey on multi-agent reinforcement learning: Coordination problems , 2010, Proceedings of 2010 IEEE/ASME International Conference on Mechatronic and Embedded Systems and Applications.

[8]  Victor R. Lesser,et al.  Learning Situation-Specific Coordination in Cooperative Multi-agent Systems , 1999, Autonomous Agents and Multi-Agent Systems.