A Bayesian approach to multiagent reinforcement learning and coalition formation under uncertainty

Sequential decision making under uncertainty is always a challenge for autonomous agents populating a multiagent environment, since their behaviour is inevitably influenced by the behaviour of others. Further, agents have to constantly struggle to find the right balance between exploiting current information regarding the environment and the rest of its inhabitants, and exploring so that they acquire additional information. Moreover, they need to profitably trade off short-term rewards with anticipated long-term ones, while learning through interaction about the environment and others—employing techniques from reinforcement learning (RL), a fundamental area of study within artificial intelligence (AI). Coalition formation is a problem of great interest within game theory and AI, allowing autonomous individually rational agents to form stable or transient teams (or coalitions) to tackle an underlying task. Agents participating in realistic scenarios of repeated coalition formation under uncertainty face the issues identified above, and need to bargain to succesfully negotiate the terms of their participation in coalitions—often having to compromise individual with team welfare effectively. In this thesis, we provide theoretical and algorithmic tools to accommodate sequential decision making under uncertainty in multiagent settings, dealing with the issues above. Specifically, we combine multiagent Bayesian RL with game theoretic ideas to facilitate the agents' sequential decision making. We deal with popular multiagent problems which were to date not tackled under uncertainty, or more specifically under type uncertainty. In our work, we assume that the environment dynamics or the types (capabilities) of other agents are not known, and thus the agents have to account for this uncertainty, in a Bayesian way, when making decisions. Handling type uncertainty allows information about others acquired within one setting to be exploited in possibly different settings in the future. The core of our contributions lies in the area of coalition formation under uncertainty. We studied several aspects of both the cooperative and non-cooperative facets of this problem, coining new theoretical concepts, proving theoretical results, presenting and evaluating algorithms for use in this context, and proposing a Bayesian RL framework for optimal repeated coalition formation under uncertainty.