Guidelines for Establishing Instantiation Validity in IT Artifacts: A Survey of IS Research

The centrality of information technology IT artifacts in Information Systems IS research makes it important to understand the relationship between artifacts and the theoretical constructs they purport to instantiate. Despite the central role of the IT artifact in IS research, there are no generally accepted principles for establishing instantiation validity --- the extent to which an artifact is a valid instantiation of a theoretical construct or a manifestation of a design principle. We survey relevant knowledge in IS and identify potential guidelines that may address threats to instantiation validity. The guidelines are intended for researchers and reviewers when using IT artifacts in theory testing and when evaluating design science artifacts.

[1]  Karlheinz Kautz,et al.  Authentic And Concurrent Evaluation -Refining An Evaluation Approach In Design Science Research , 2011, PACIS.

[2]  Jan Pries-Heje,et al.  A Comprehensive Framework for Evaluation in Design Science Research 1 , 2022 .

[3]  S. Chatterjee,et al.  Design Science Research in Information Systems , 2010 .

[4]  Ofer Arazy,et al.  A Theory-Driven Design Framework for Social Recommender Systems , 2010, J. Assoc. Inf. Syst..

[5]  Alan R. Hevner,et al.  Design Research in Information Systems: Theory and Practice , 2010 .

[6]  Omar El Sawy,et al.  Building an Information System Design Theory for Vigilant EIS , 1992, Inf. Syst. Res..

[7]  Roman Lukyanenko,et al.  A Survey of Cognitive Theories to Support Data Integration , 2011, AMCIS.

[8]  Alan R. Hevner,et al.  Focus Groups for Artifact Refinement and Evaluation in Design Research , 2010, Commun. Assoc. Inf. Syst..

[9]  Ron Weber,et al.  Guidelines for Empirical Evaluations of Conceptual Modeling Grammars , 2009, J. Assoc. Inf. Syst..

[10]  Gerardine DeSanctis,et al.  ICIS Paper: Methodological Issues in Experimental IS Research: Experiences and Recommendations , 1985, MIS Q..

[11]  Shirley Gregor,et al.  Prescriptive Knowledge in IS Research: Conceptualizing Design Principles in Terms of Materiality, Action, and Boundary Conditions , 2015, 2015 48th Hawaii International Conference on System Sciences.

[12]  Sandeep Purao,et al.  Action Design Research , 2011, MIS Q..

[13]  Sandeep Purao,et al.  Truth or Dare: The Ontology Question in Design Science Research , 2013, J. Database Manag..

[14]  Detmar W. Straub,et al.  Validation Guidelines for IS Positivist Research , 2004, Commun. Assoc. Inf. Syst..

[15]  Monica Chiarini Tremblay,et al.  Advancing the Impact of Design Science: Moving from Theory to Practice , 2014, Lecture Notes in Computer Science.

[16]  Viswanath Venkatesh,et al.  Bridging the Qualitative-Quantitative Divide: Guidelines for Conducting Mixed Methods Research in Information Systems , 2013, MIS Q..

[17]  J. G. Snodgrass,et al.  A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. , 1980, Journal of experimental psychology. Human learning and memory.

[18]  Shirley Gregor,et al.  Causality: the elephant in the room in information systems epistemology , 2011, ECIS.

[19]  Jan vom Brocke,et al.  Design Science at the Intersection of Physical and Virtual Design , 2013, Lecture Notes in Computer Science.

[20]  Izak Benbasat,et al.  Development of an Instrument to Measure the Perceptions of Adopting an Information Technology Innovation , 1991, Inf. Syst. Res..

[21]  Vijay K. Vaishnavi,et al.  A Framework for Theory Development in Design Science Research: Multiple Perspectives , 2012, J. Assoc. Inf. Syst..

[22]  Roman Lukyanenko,et al.  Reconciling Theories with Design Choices in Design Science Research , 2013, DESRIST.

[23]  Detmar W. Straub,et al.  Validation in Information Systems Research: A State-of-the-Art Assessment , 2001, MIS Q..

[24]  Roman Lukyanenko,et al.  The IQ of the Crowd: Understanding and Improving Information Quality in Structured User-Generated Content , 2014, Inf. Syst. Res..

[25]  Alan R. Hevner,et al.  Design Research in Information Systems , 2010 .

[26]  Roman Lukyanenko,et al.  Instantiation Validity in IS Design Research , 2014, DESRIST.

[27]  Jan vom Brocke,et al.  Evaluations in the Science of the Artificial - Reconsidering the Build-Evaluate Pattern in Design Science Research , 2012, DESRIST.

[28]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[29]  Robert Winter,et al.  Fail Early, Fail Often: Towards Coherent Feedback Loops in Design Science Research Evaluation , 2014, ICIS.

[30]  Izak Benbasat,et al.  The Effects of Personalizaion and Familiarity on Trust and Adoption of Recommendation Agents , 2006, MIS Q..