Effects of Communication in Cooperative Q-Learning

Reinforcement learning has been utilized to investigate the distributed learn- ing problem in many different multi-agent and team-based scenarios. Invariably, using a distributed learning approach by allowing agents to exchange what they have learned, by either direct communication or by episodic exchanges, the team does better in terms of achieving its goals. The cause-effect relationship between communication and improved performance has been studied in a number of different team based scenarios; however, it is difficult to find a detailed analysis of the effect the communication has as a function of time on the elements of the simulation. Of particular interest is the effect on the learning of agents when part of a communicating multi-agent system. Studies show the effects of cooperative learning in the form of accelerated group learning rates; but it is difficult to pinpoint through the literature exactly how this observed effect is realized within individ- ual agents over time. This paper investigates the effect of communication as a function of time on a multi-agent simulation based on a military distillation, utilizing a modified Q-learning algorithm.