Contributions of In Virtuo and In Silico Experiments for the Future of Empirical Studies in Software Engineering

Empirical software engineers usually use a two-staged taxonomy based on the terms in vivo or in vitro, according the control level that can be attained upon the environment where these studies are executed. Despite its importance, this taxonomy does not capture relevant issues concerned with different study categories, mainly those exploring computer models. We faced this limitation when classifying experiments regarding software project management in which we were involved. By analyzing other sciences that also apply experiments to support their research; we observe that the in vivo/in vitro taxonomy has been extended to accommodate the use of computer models. Two new experiment classes (namely in virtuo and in silico) complement the original taxonomy. They regard the use of computer models to simulate the environment, the object under analysis and the subjects that take part in an experimental study. When applied to the software engineering experiments, this four-staged taxonomy helped us to classify our studies, revealing additional experiment features not observed for the conventional studies yet. In this paper we propose the use of this four-staged taxonomy for categorizing software engineering experiments and evaluate some implications of using in virtuo and in silico experiments in software engineering. Besides, based on the possible contributions such studies can bring, we propose some topics to compose a research agenda for the future of empirical software engineering.