An Information System Agnostic Environment for Reproducible Experiments

Designing reproducible experiments with information systems (IS) is a challenging task, mainly due to the complex nature of processes, vendor specific tools and the difficulty to find an adequate abstraction level. While technical challenges (e.g. finding performance bottlenecks) can be investigated in detail using simplified near-life settings or purposive research prototypes, these approaches are not feasible in experiments concerned with IS used in complex environments and problem domains (e.g. governance, risk and compliance management) as the software often cannot be simplified without considerable effort or losing rigor. On the other hand, software used in such domains has almost never been designed by the vendors with researchers’ goals and needs in mind which makes their usage in experiments burdensome. To address this, we present the research prototype of an experimentation environment that simplifies the creation of reproducible experiments around off-the-shelf information systems.