Using DCOPs to Balance Exploration and Exploitation in Time-Critical Domains

Substantial work has investigated balancing exploration and exploita- tion, but relatively little has addressed this tradeoff in the context of coordinated multi-agent interactions. This paper introduces a class of problems in which agents must maximize their on-line reward, a decomposable function dependent on pairs of agent's decisions. Unlike previous work, agents must both learn the reward function and exploit it on-line, critical properties for a class of physically- motivated systems, such as mobile wireless networks. This paper introduces algo- rithms motivated by the Distributed Constraint Optimization Problem framework and demonstrates when, and at what cost, increasing agents' coordination can im- prove the global reward on such problems.