SwarmRob: A Docker-Based Toolkit for Reproducibility and Sharing of Experimental Artifacts in Robotics Research

Due to the high complexity, the reproducibility of experiments is one of the fundamental problems of robotics research. Although the problem has been identified by the community, the approaches to solve it are limited. The toolkit presented in this paper attempts to provide an approach to address the problem of reproducibility and the associated exchange of experimental artifacts. In order to simplify this process, the toolkit supports both the initial definition of experiments and the subsequent reproduction through a new type of workflow based on dynamic hardware-dependent assignment of services to devices and their automatic initialization. The evaluation of the system shows a high performance compared to a native solution as well as a substantial time saving compared to the manual set-up of an experiment.

[1]  Raj Jain,et al.  A Quantitative Measure Of Fairness And Discrimination For Resource Allocation In Shared Computer Systems , 1998, ArXiv.

[2]  Andrew J. Davison,et al.  A benchmark for RGB-D visual odometry, 3D reconstruction and SLAM , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[3]  Eugenio Guglielmelli.  Research Reproducibility and Performance Evaluation for Dependable Robots [From the Editor's Desk] , 2015, IEEE Robotics Autom. Mag..

[4]  Lawrence Kreeger,et al.  Virtual eXtensible Local Area Network (VXLAN): A Framework for Overlaying Virtualized Layer 2 Networks over Layer 3 Networks , 2014, RFC.

[5]  Geoffrey C. Fox,et al.  GPU Passthrough Performance: A Comparison of KVM, Xen, VMWare ESXi, and LXC for CUDA and OpenCL Applications , 2014, 2014 IEEE 7th International Conference on Cloud Computing.

[6]  Angel P. del Pobil,et al.  Toward Replicable and Measurable Robotics Research [From the Guest Editors] , 2015, IEEE Robotics Autom. Mag..

[7]  Li Wang,et al.  The Robotarium: A remotely accessible swarm robotics research testbed , 2016, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Aljoscha Pörtner,et al.  Ambient Assisted Robot Object Search , 2017, ICOST.

[9]  Stéphane Doncieux,et al.  The Transferability Approach: Crossing the Reality Gap in Evolutionary Robotics , 2013, IEEE Transactions on Evolutionary Computation.

[10]  Carl Boettiger,et al.  An introduction to Docker for reproducible research , 2014, OPSR.

[11]  Vijay Kumar,et al.  Experimental Testbed for Large Multirobot Teams , 2008, IEEE Robotics Autom. Mag..

[12]  V G Andrew,et al.  AN EFFICIENT IMPLEMENTATION OF A SCALING MINIMUM-COST FLOW ALGORITHM , 1997 .

[13]  Andrew W. Fitzgibbon,et al.  KinectFusion: Real-time dense surface mapping and tracking , 2011, 2011 10th IEEE International Symposium on Mixed and Augmented Reality.

[14]  Magnus Egerstedt,et al.  The GRITSBot in its natural habitat - A multi-robot testbed , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[15]  Robert Ricci,et al.  Mobile Emulab: A Robotic Wireless and Sensor Network Testbed , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[16]  Michael F. P. O'Boyle,et al.  Introducing SLAMBench, a performance and accuracy benchmarking methodology for SLAM , 2014, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Sven Wachsmuth,et al.  The Cognitive Interaction Toolkit - Improving Reproducibility of Robotic Systems Experiments , 2014, SIMPAR.

[18]  Harald C. Gall,et al.  Using Docker Containers to Improve Reproducibility in Software Engineering Research , 2016, 2016 IEEE/ACM 38th International Conference on Software Engineering Companion (ICSE-C).

[19]  A. Stubbs,et al.  Multivehicle systems control over networks: a hovercraft testbed for networked and decentralized control , 2006, IEEE Control Systems.

[20]  Simha Sethumadhavan,et al.  RoboBench: Towards sustainable robotics system benchmarking , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[21]  Aljoscha Pörtner,et al.  Gesture-Based Object Localization for Robot Applications in Intelligent Environments , 2018, 2018 14th International Conference on Intelligent Environments (IE).