The RoboCup Logistics League as a Holistic Multi-Robot Smart Factory Benchmark

With autonomous mobile robots receiving increased attention in industrial contexts, the need for benchmarks becomes more and more an urgent matter. The RoboCup Logistics League (RCLL) is one specific industry-inspired scenario focusing on production logistics within a Smart Factory. In this paper, we describe how the RCLL allows to assess the performance of a group of robots within the scenario as a whole, focusing specifically on the coordination and cooperation strategies and the methods and components to achieve them. We report on recent efforts to analyze performance of teams in 2014 to understand the implications of the current grading scheme, and derived criteria and metrics for performance assessment based on Key Performance Indicators (KPI) adapted from classic factory evaluation. We reflect on differences and compatibility towards RoCKIn, a recent major benchmarking European project.