Decentralised Online Planning for Multi-Robot Warehouse Commissioning

Warehouse commissioning is a complex task in which a team of robots needs to gather and deliver items as fast and efficiently as possible while adhering to the constraint capacity of the robots. Typical centralised control approaches can quickly become infeasible when dealing with many robots. Instead, we tackle this spatial task allocation problem via distributed planning on each robot in the system. State of the art distributed planning approaches suffer from a number of limiting assumptions and ad-hoc approximations. This paper demonstrates how to use Monte Carlo Tree Search (MCTS) to overcome these limitations and provide scalability in a more principled manner. Our simulation-based evaluation demonstrates that this translates to higher task performance, especially when tasks get more complex. Moreover, this higher performance does not come at the cost of scalability: in fact, the proposed approach scales better than the previous best approach, demonstrating excellent performance on an 8-robot team servicing a warehouse comprised of over 200 locations.

[1]  Rémi Coulom,et al.  Efficient Selectivity and Backup Operators in Monte-Carlo Tree Search , 2006, Computers and Games.

[2]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[3]  David Silver,et al.  Combining online and offline knowledge in UCT , 2007, ICML '07.

[4]  Martin L. Puterman,et al.  Markov Decision Processes: Discrete Stochastic Dynamic Programming , 1994 .

[5]  Mark H. M. Winands,et al.  Monte Carlo Tree Search for the Hide-and-Seek Game Scotland Yard , 2012, IEEE Transactions on Computational Intelligence and AI in Games.

[6]  Elizabeth Sklar,et al.  Auction-Based Task Allocation for Multi-robot Teams in Dynamic Environments , 2015, TAROS.

[7]  Yishay Mansour,et al.  A Sparse Sampling Algorithm for Near-Optimal Planning in Large Markov Decision Processes , 1999, Machine Learning.

[8]  Gerhard Wäscher,et al.  Order Batching in Order Picking Warehouses: A Survey of Solution Approaches , 2012 .

[9]  Aníbal Ollero,et al.  Decentralized multi-robot cooperation with auctioned POMDPs , 2012, 2012 IEEE International Conference on Robotics and Automation.

[10]  Han-Lim Choi,et al.  Consensus-Based Decentralized Auctions for Robust Task Allocation , 2009, IEEE Transactions on Robotics.

[11]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[12]  Edmund H. Durfee,et al.  A decision-theoretic characterization of organizational influences , 2012, AAMAS.

[13]  Maria L. Gini,et al.  Monte Carlo Tree Search with Branch and Bound for Multi-Robot Task Allocation , 2016 .

[14]  Wiebe van der Hoek,et al.  Effective Approximations for Multi-Robot Coordination in Spatially Distributed Tasks , 2015, AAMAS.

[15]  Peter Auer,et al.  Finite-time Analysis of the Multiarmed Bandit Problem , 2002, Machine Learning.

[16]  Karl Tuyls,et al.  Collision avoidance under bounded localization uncertainty , 2012, 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[17]  Steven Okamoto,et al.  Dynamic Multi-Agent Task Allocation with Spatial and Temporal Constraints , 2014, AAAI.

[18]  Frederik Hegger,et al.  RoboCup@Work: Competing for the Factory of the Future , 2014, RoboCup.

[19]  Maja J. Mataric,et al.  Sold!: auction methods for multirobot coordination , 2002, IEEE Trans. Robotics Autom..

[20]  Nidhi Kalra,et al.  Market-Based Multirobot Coordination: A Survey and Analysis , 2006, Proceedings of the IEEE.

[21]  Csaba Szepesvári,et al.  Bandit Based Monte-Carlo Planning , 2006, ECML.

[22]  Edmund H. Durfee,et al.  Resource-Driven Mission-Phasing Techniques for Constrained Agents in Stochastic Environments , 2010, J. Artif. Intell. Res..

[23]  Alan Fern,et al.  Lower Bounding Klondike Solitaire with Monte-Carlo Planning , 2009, ICAPS.

[24]  Maja J. Mataric,et al.  Multi-robot task allocation: analyzing the complexity and optimality of key architectures , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[25]  Craig Boutilier,et al.  Planning, Learning and Coordination in Multiagent Decision Processes , 1996, TARK.

[26]  Simon M. Lucas,et al.  A Survey of Monte Carlo Tree Search Methods , 2012, IEEE Transactions on Computational Intelligence and AI in Games.