Uncertainty handling and decision making in multi-agent cooperation

An autonomous decision maker, such as an intelligent agent, must make decisions in the presence of uncertainty. Furthermore, in a multi-agent system where the agents are distributed, agents need to deal with not only uncertain outcomes of local events but also uncertainty associated with events happening in other agents in order to maintain proper coordination of the activities of the agents. This dissertation focuses on the problem of handling uncertainty and snaking decisions related to agent coordination in cooperative multi-agent systems. Our hypothesis is that the choice of coordination strategies must take into account the specific characteristics of the environments in which the agents operate in order to improve performance. Our goal is to provide a quantitative model and a set of tools and methodologies that can be used in evaluating and developing situation specific coordination strategies, to model uncertainty in coordination, and to facilitate understanding which information is necessary when making coordination decisions. Our approach is first to examine the types of uncertainty that need to be considered when making coordination decisions, and then to incorporate them explicitly in the decision making. The result is a richer semantics of agent commitments that quantitatively represent the possible effects of uncertain events, and we demonstrate its performance through simulation with a heuristic scheduler. We then move away from heuristic problem solving and establish a formal decision-theoretic framework for multi-agent decision making. We call this framework decentralized multi-agent Markov decision processes . It categorizes agent decisions into action decisions and communication decisions, and we experiment with communication decisions to demonstrate how the performance of different coordination strategies varies according to the environment parameters. Finally, to address the problem of complexity in solving the decision processes we have defined, and to provide a connection between centralized policies and decentralized policies, we develop a methodology for generating a set of decentralized multi-agent policies based on solving the centralized multi-agent Markov decision process. We study its performance by comparing it to heuristic policies and show how to reduce communication costs.