Applying Evaluations While Building the Artifact -- Experiences from the Development of Process Model Complexity Metrics

The Design Science Research method is decisive for the quality of the resulting solution. Thus, many discussions focus the evaluation of the solution at the end of the Design Science cycle. But design, implementation and evaluation of artifacts are laborious and need to be repeated if the artifact does not meet the evaluation criteria. Thus, recent works have proposed to conduct additional evaluations early in the Design Science process to possibly reduce the number of repetitions of the research process. However, such early evaluations may also be an unnecessary burden. Therefore, this work presents a case where these additional evaluations are applied ex-post in a practical research project which developed process model complexity metrics and the outcomes are compared. Once compared, benefits and limitations of early evaluations are discussed.