Model-Based Performance Testing for Robotics Software Components

In complex technical systems like robotics platforms, a manifold of issues can impair their dependability. While common testing and simulation methods largely focus on functional aspects, the utilization of resources like CPU, network bandwidth, or memory is only rarely tested systematically. With this contribution we propose a novel Domain-Specific Language (DSL) for modeling performance tests for individual robotics components with the aim to establish a systematic testing process for detecting regressions regarding the resource utilization. This DSL builds upon a testing framework from previous research and aims to significantly reduce the effort and complexity for creating performance tests. The DSL is built using the MPS language workbench and provides a feature-rich editor with modern editing aids. An evaluation indicates that developing performance tests requires only one third of the work in comparison to the original Java-based API.

[1]  Sebastian Wrede,et al.  Results of the Survey: Failures in Robotics and Intelligent Systems , 2017, ArXiv.

[2]  Nabor das Chagas Mendonça,et al.  A Declarative Environment for Automatic Performance Evaluation in IaaS Clouds , 2013, 2013 IEEE Sixth International Conference on Cloud Computing.

[3]  Ahmed E. Hassan,et al.  A Survey on Load Testing of Large-Scale Software Systems , 2015, IEEE Transactions on Software Engineering.

[4]  Markus Völter,et al.  Language and IDE Modularization and Composition with MPS , 2011, GTTSE.

[5]  Sebastian Wrede,et al.  Performance regression testing and run-time verification of components in robotics systems , 2017, Adv. Robotics.

[6]  Liming Zhu,et al.  Benchmark Generation Using Domain Specific Modeling , 2007, 2007 Australian Software Engineering Conference (ASWEC'07).

[7]  Patrizia Scandurra,et al.  Component-based robotic engineering (Part I) [Tutorial] , 2009, IEEE Robotics & Automation Magazine.

[8]  Sebastian Wrede,et al.  A Survey on Domain-specific Modeling and Languages in Robotics , 2016 .

[9]  John Zic,et al.  Yet Another Performance Testing Framework , 2008 .

[10]  Darren Sawyer,et al.  A little language for rapidly constructing automated performance tests , 2011, ICPE '11.

[11]  Sebastian Wrede,et al.  Autonomous fault detection for performance bugs in component-based robotic systems , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[12]  Sebastian Wrede,et al.  Evaluating a Graph Query Language for Human-Robot Interaction Data in Smart Environments , 2017, STAF Workshops.

[13]  Gerald Steinbauer,et al.  A Survey about Faults of Robots Used in RoboCup , 2012, RoboCup.

[14]  Douglas C. Schmidt,et al.  ROAR: A QoS-oriented modeling framework for automated cloud resource allocation and optimization , 2016, J. Syst. Softw..

[15]  Eelco Visser,et al.  DSL Engineering - Designing, Implementing and Using Domain-Specific Languages , 2013 .

[16]  Morgan Quigley,et al.  ROS: an open-source Robot Operating System , 2009, ICRA 2009.

[17]  Avelino Francisco Zorzo,et al.  Performance testing modeling: an empirical evaluation of DSL and UML-based approaches , 2016, SAC.

[18]  Sebastian Wrede,et al.  A Meta-model and Toolchain for Improved Interoperability of Robotic Frameworks , 2012, SIMPAR.

[19]  Sebastian Wrede,et al.  Domain-Specific Language Modularization Scheme Applied to a Multi-Arm Robotics Use-Case , 2017 .

[20]  Wilhelm Hasselbring,et al.  Automatic Extraction of Probabilistic Workload Specifications for Load Testing Session-Based Application Systems , 2015, EAI Endorsed Trans. Self Adapt. Syst..

[21]  Calton Pu,et al.  Expertus: A Generator Approach to Automate Performance Testing in IaaS Clouds , 2012, 2012 IEEE Fifth International Conference on Cloud Computing.

[22]  Avelino Francisco Zorzo,et al.  Canopus: A Domain-Specific Language for Modeling Performance Testing , 2016, 2016 IEEE International Conference on Software Testing, Verification and Validation (ICST).