Model-based integration testing of ROS packages: A mobile robot case study

We apply model-based testing - a black box testing technology - to improve the state of the art of integration testing of navigation and localisation software for mobile robots built in ROS. Online model-based testing involves building executable models of the requirements and executing them in parallel with the implementation under test (IUT). In the current paper we present an automated approach to generating a model from the topological map that specifies where the robot can move to. In addition, we show how to specify scenarios of interest and how to add human models to the simulated environment according to a specified scenario. We measure the quality of the tests by code coverage, and empirically show that it is possible to achieve increased test coverage by specifying simple scenarios on the automatically generated model of the topological map. The scenarios augmented by adding humans to specified rooms at specified stages of the scenario simulate the changes in the environment caused by humans. Since we test navigation at coordinate and topological level, we report on finding problems related to the topological map.

[1]  Valiallah Monajjemi,et al.  Drums: A Middleware-Aware Distributed Robot Monitoring System , 2014, 2014 Canadian Conference on Computer and Robot Vision.

[2]  Marc Hanheide,et al.  Long-term topological localisation for service robots in dynamic environments using spectral maps , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Wang Yi,et al.  Timed Automata: Semantics, Algorithms and Tools , 2003, Lectures on Concurrency and Petri Nets.

[4]  Sebastian G. Elbaum,et al.  Reducing failure rates of robotic systems though inferred invariants monitoring , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Kim G. Larsen,et al.  Testing Real-Time Systems Using UPPAAL , 2008, Formal Methods and Testing.

[6]  Martin Heckmann,et al.  Online data-driven fault detection for robotic systems , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Nick Hawes,et al.  Using Qualitative Spatial Relations for indirect object search , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[9]  Simon Lacroix,et al.  Simulating Complex Robotic Scenarios with MORSE , 2012, SIMPAR.

[10]  Heinz Wörn,et al.  Robot Unit Testing , 2014, SIMPAR.

[11]  Rares Ambrus,et al.  Meta-rooms: Building and maintaining long term spatial models in a dynamic world , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[12]  Aivo Anier,et al.  Model based Continual Planning and Control for Assistive Robots , 2012, HEALTHINF.

[13]  David S. Rosenblum,et al.  Known unknowns: testing in the presence of uncertainty , 2014, SIGSOFT FSE.

[14]  Marc Hanheide,et al.  Now or later? Predicting and maximising success of navigation actions from long-term experience , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[15]  John Folkesson,et al.  Combining Top-down Spatial Reasoning and Bottom-up Object Class Recognition for Scene Understanding , 2014, IROS 2014.