Using Multi-Agent Options to Reduce Learning Time in Reinforcement Learning

Distributed multi-agent learning has recently received significant interest but also proven to be very complex as the decisions made by any individual agent are not the only factors in the outcomes of those decisions. Uncertainty associated in the decisions and exploration choices of other agents add complexity and delay to individual learning processes. To address this complexity and provide for better scaling of distributed multi-agent learning this paper extends the options framework and the Nash-Q learning technique to apply to multi-agent distributed learning in non-cooperative game theoretic settings. We illustrate the effectiveness of this approach in a grid world and demonstrate improved learning for a set of tasks in a semi-cooperative environment.