Design alternatives for the evaluation of design science research artifacts

Within a consideration of cost effectiveness the evaluation of design science research artifacts is of major importance. In the past, a plenitude of approaches has been developed for this purpose -- partly artifact-specific, partly artifact-neutral. Nonetheless, there is a lack of a comprehensive overview over existing methods as well as a systemization of those with regard to fundamental structuring criteria. The paper at hand surveys existing methods and introduces a framework that equally supports the designer and the user of artifact evaluation approaches. Subsequent to the embedding of the framework into the design science research process two exemplary application scenarios are described.

[1]  Bhola,et al.  Evaluating “Literacy for Development” Projects, Programmes and Campaigns: Evaluation Planning, Design and Implementation, and Utilization of Evaluation Results , 1990 .

[2]  Björn Niehaves,et al.  Evaluation of Conceptual Models - A Structuralist Approach , 2005, ECIS.

[3]  Shirley Gregor,et al.  The Nature of Theory in Information Systems , 2006, MIS Q..

[4]  Donald R. Strand,et al.  The business of engineering , 2005 .

[5]  Ulrich Frank,et al.  Evaluation von Artefakten in der Wirtschaftsinformatik , 1999 .

[6]  Brian T. Pentland,et al.  Process Grammar as a Tool for Business Process Design , 2008, MIS Q..

[7]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[8]  Rudy Hirschheim,et al.  A paradigmatic and methodological examination of information systems research from 1991 to 2001 , 2004, Inf. Syst. J..

[9]  Martin Bichler,et al.  Design science in information systems research , 2006, Wirtschaftsinf..

[10]  N. Cross.  Designerly Ways of Knowing: Design Discipline Versus Design Science , 2001, Design Issues.

[11]  Paul Veerkamp,et al.  Modeling Design Process , 1990, AI Mag..

[12]  Vijay K. Vaishnavi,et al.  Theory Development in Design Science Research: Anatomy of a Research Project , 2008 .

[13]  Samir Chatterjee,et al.  A Design Science Research Methodology for Information Systems Research , 2008 .

[14]  John R. Venable,et al.  The role of theory and theorising in Design Science research , 2006 .

[15]  Yair Wand,et al.  Using Cognitive Principles to Guide Classification in Information Systems Modeling , 2008, MIS Q..

[16]  T. Ritchey,et al.  Problem structuring using computer-aided morphological analysis , 2006, J. Oper. Res. Soc..

[17]  Keng Siau,et al.  Evaluation techniques for systems analysis and design modelling methods – a review and comparative analysis , 2011, Inf. Syst. J..

[18]  Jan Pries-Heje,et al.  Strategies for Design Science Research Evaluation , 2008, ECIS.

[19]  Peter Loos,et al.  Multiperspective Evaluation of Reference Models - Towards a Framework , 2003, ER.

[20]  Robert Winter,et al.  Design science research in Europe , 2008 .

[21]  Michael Scriven,et al.  The new science of evaluation , 1998 .

[22]  H. S. Bhola,et al.  Evaluating "Literacy for Development" Projects, Programs and Campaigns. Evaluation Planning, Design and Implementation, and Utilization of Evaluation Results. UIE Handbooks and Reference Books 3. , 1990 .

[23]  R. Winter,et al.  Business Engineering: Auf dem Weg zum Unternehmen des Informationszeitalters , 2000 .

[24]  Sven A. Carlsson,et al.  Towards an Information Systems Design Research Framework: A Critical Realist Perspective , 2006 .

[25]  Veda C. Storey,et al.  Design science in the information systems discipline: an introduction to the special issue on design science research , 2008 .

[26]  Björn Niehaves,et al.  Epistemological perspectives on IS research: a framework for analysing and systematizing epistemological assumptions , 2007, Inf. Syst. J..

[27]  Reinhard Stockmann,et al.  Evaluation und Qualitätsentwicklung : eine Grundlage für wirkungsorientiertes Qualitätsmanagement , 2006 .

[28]  Omar A. El Sawy,et al.  Assessing Information System Design Theory in Perspective: How Useful Was our 1992 Initial Rendition? , 2004 .

[29]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[30]  Salvatore T. March,et al.  Design and natural science research on information technology , 1995, Decis. Support Syst..

[31]  Ron Weber,et al.  An Ontological Model of an Information System , 1990, IEEE Trans. Software Eng..

[32]  Sjaak Brinkkemper,et al.  Meta-Modelling Based Assembly Techniques for Situational Method Engineering , 1999, Inf. Syst..

[33]  Christian Riege,et al.  Evaluation in der gestaltungsorientierten Wirtschaftsinformatik - Systematisierung nach Erkenntnisziel und Gestaltungsziel , 2008 .

[34]  Ulrich Frank,et al.  Evaluation of Reference Models , 2006 .