The case for design science utility and quality - Evaluation of design science artifact within the sustainable ICT capability maturity framework

In Design Science Research (DSR), evaluation of research outputs in form of design artifacts has been discussed in numerous publications. Many researchers have emphasised the criteria of utility for design artifacts, whereas recent approaches have extended this view to other criteria such as efficiency, consistency, accuracy, performance and reliability to mention a few. In this paper we revisit the evaluation discussion in design science and describe a practical oriented evaluation framework. In order to incorporate the evaluation along on-going design cycles, we propose to build on work related to semiotic and information quality. We argue that design science research is usually complex and requires a more detailed evaluation approach. We review literature and follow a semiotic framework for managing knowledge in complex environments. Together with well established information quality criteria we demonstrate that the proposed framework can help to provide a practical evaluation approach within a complex design environment. The framework was developed within the context of a novel IT Management model. We describe its development during the creation of the maturity model for Sustainable Information and Communication Technology (ICT) called the SICTCapability Maturity Framework (SICT-CMF). The context was selected as it is particular interesting with design artifacts created within an open and complex innovation community. The research acknowledges the importance of the utility criteria, but also includes other criteria such as artifact quality, consistency and accuracy in form of a more differentiated view of design science evaluation.