Multi-robot coordination and competition using mixed integer and linear programs

Markov decision processes (MDPs) and partially observable Markov decision processes (POMDPs) are preferred methods representing complex uncertain dynamic systems and determining an optimal control policy to manipulate the system in the desired manner. Until recently, controlling a system composed of multiple agents using the MDP methodology was impossible due to an exponential increase in the size of the MDP problem representation. In this thesis, a novel method for solving large multi-agent MDP systems is presented which avoids this exponential size increase while still providing optimal policies for a large class of useful problems. This thesis provides the following main contributions: A novel description language for multi-agent MDPs. We develop two different modeling techniques for representing multi-agent MDP (MAMDP) coordination problems. The first phrases the problem using a linear program which avoids the exponential state space size of multi-agent MDPs. The second, more expressive representation, expands upon the linear programming representation with the addition of integer constraints. For many problems, these representations exactly represent the original problem with exponentially fewer variables and constraints. This leads to an efficient and optimal solution of the multi-agent MDP. A novel multi-agent coordination method for multi-agent MDPs . We use the Dantzig-Wolfe decomposition technique and the branch and bound method to solve the above models efficiently. These solution methods overcome significant drawbacks in related work. We develop a multi-robot towing and foraging model, devise a novel multi-agent path planner and solve coordination problems with quadratic cost on a global resource. A method to determine the optimal strategies for competing teams . One team of players is allowed to determine the cost function of the MAMDP of the second team. This allows us to solve a zero-sum game played by both teams. One team chooses cost functions and the other coordinates to find the least cost plan given the other team's possible strategies. Each team is coordinated using the newly developed multi-agent coordination method above. A game of robot paintball. Using the above techniques we solve a game of multi-robot paintball. This game is played with real robots at high speeds.