Hierarchical multi-agent reinforcement learning