On Experiments in Design Science Research and Theory Development: A Literature Review

Experimental research is an established method in Information Systems Research and well suited for Design Science Research (DSR). Our review of 71 published research articles aims at contributing to the DSR body of knowledge by systematically collecting the dependent and independent variables experimentally tested. Here, we identify a lack of cumulative research in DSR as only very few theory constructs are (re-)used. The paper provides several implications for theory and an outlook on potentially fruitful avenues for future research.

[1]  Izak Benbasat,et al.  Research Note - A Contingency Approach to Investigating the Effects of User-System Interaction Modes of Online Decision Aids , 2013, Inf. Syst. Res..

[2]  Mark A. Fuller,et al.  Designing Interfaces with Social Presence: Using Vividness and Extraversion to Create Social Recommendation Agents , 2009, J. Assoc. Inf. Syst..

[3]  Leonard Adelman,et al.  Experiments, quasi-experiments, and case studies: A review of empirical methods for evaluating decision support systems , 1991, IEEE Trans. Syst. Man Cybern..

[4]  René Riedl,et al.  Are There Neural Gender Differences in Online Trust? An fMRI Study on the Perceived Trustworthiness of eBay Offers , 2010, MIS Q..

[5]  Omar El Sawy,et al.  Building an Information System Design Theory for Vigilant EIS , 1992, Inf. Syst. Res..

[6]  Robert P. Bostrom,et al.  An Investigation of the Appropriation of Technology-Mediated Training Methods Incorporating Enactive and Collaborative Learning , 2013, Inf. Syst. Res..

[7]  Milena M. Head,et al.  Exploring human images in website design: a multi-method approach , 2009 .

[8]  Samir Chatterjee,et al.  A Design Science Research Methodology for Information Systems Research , 2008 .

[9]  Alessandro Acquisti,et al.  The Effect of Online Privacy Information on Purchasing Behavior: An Experimental Study , 2011, WEIS.

[10]  H. Simon,et al.  The sciences of the artificial (3rd ed.) , 1996 .

[11]  Nigel King,et al.  Using templates in the thematic analysis of text , 2004 .

[12]  Yair Wand,et al.  Guidelines for Designing Visual Ontologies to Support Knowledge Identification , 2011, MIS Q..

[13]  S. Carlsson.  Design Science Research in Information Systems: A Critical Realist Approach , 2010 .

[14]  Salvatore T. March,et al.  Design and natural science research on information technology , 1995, Decis. Support Syst..

[15]  Robert Winter,et al.  On the Use of Experiments in Design Science Research: A Proposition of an Evaluation Framework , 2014, Commun. Assoc. Inf. Syst..

[16]  Young Eun Lee,et al.  The Effects of Virtual Reality on Consumer Learning: An Empirical Investigation , 2005, MIS Q..

[17]  John W. Creswell,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2010 .

[18]  Chuan-Hoo Tan,et al.  Using Personal Communication Technologies for Commercial Communications: A Cross-Country Investigation of Email and SMS , 2014, Inf. Syst. Res..

[19]  Gerald Häubl,et al.  When Social Media Can Be Bad for You: Community Feedback Stifles Consumer Creativity and Reduces Satisfaction with Self-Designed Products , 2013, Inf. Syst. Res..

[20]  Izak Benbasat,et al.  Interactive Decision Aids for Consumer Decision Making in E-Commerce: The Influence of Perceived Strategy Restrictiveness , 2009, MIS Q..

[21]  Angelika Dimoka,et al.  On the Use of Neuropyhsiological Tools in IS Research: Developing a Research Agenda for NeuroIS , 2012, MIS Q..

[22]  Steven Walczak,et al.  Dressing Your Online Auction Business for Success: An Experiment Comparing Two eBay Businesses , 2008, MIS Q..

[23]  Shuk Ying Ho,et al.  Web Personalization as a Persuasion Strategy: An Elaboration Likelihood Model Perspective , 2005, Inf. Syst. Res..

[24]  Jingjing Li,et al.  A Transdisciplinary Approach to Construct Search and Integration , 2010, AMCIS.

[25]  Izak Benbasat,et al.  Let's Shop Online Together: An Empirical Investigation of Collaborative Online Shopping Support , 2010, Inf. Syst. Res..

[26]  Jane Webster,et al.  Enhancing the Design of Web Navigation Systems: The Influence of User Disorientation on Engagement and Performance , 2006, MIS Q..

[27]  Oliver Hinz,et al.  The Impact of Information Diffusion on Bidding Behavior in Secret Reserve Price Auctions , 2008, Inf. Syst. Res..

[28]  Vijay K. Vaishnavi,et al.  So, Talk to Me: The Effect of Explicit Goals on the Comprehension of Business Process Narratives , 2006, MIS Q..

[29]  Shirley Gregor,et al.  The Anatomy of a Design Theory , 2007, J. Assoc. Inf. Syst..

[30]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[31]  Alan R. Hevner,et al.  Design Research in Information Systems: Theory and Practice , 2010 .

[32]  Tamás D. Gedeon,et al.  Neuroscience and a Nomological Network for the Understanding and Assessment of Emotions in Information Systems Research , 2014, J. Manag. Inf. Syst..

[33]  R. M. Yaremko,et al.  Handbook of Research and Quantitative Methods in Psychology: For Students and Professionals , 1986 .

[34]  Rudy Hirschheim,et al.  A paradigmatic and methodological examination of information systems research from 1991 to 2001 , 2004, Inf. Syst. J..

[35]  Herbert A. Simon,et al.  The Sciences of the Artificial , 1970 .

[36]  Kai H. Lim,et al.  Brand Positioning Strategy Using Search Engine Marketing , 2010, MIS Q..

[37]  Starr Roxanne Hiltz,et al.  An Assessment of Group Support Systems Experimental Research: Methodology and Results , 1998, J. Manag. Inf. Syst..

[38]  Judy McKay,et al.  A review of design science in information systems , 2005 .

[39]  Matthew L. Jensen,et al.  Evaluation of Competing Candidate Solutions in Electronic Networks of Practice , 2014, Inf. Syst. Res..

[40]  Ritu Agarwal,et al.  Practicing Safe Computing: A Multimedia Empirical Examination of Home Computer User Security Behavioral Intentions , 2010, MIS Q..

[41]  Izak Benbasat,et al.  HCI Research: Future Challenges and Directions , 2010 .

[42]  Izak Benbasat,et al.  The Effects of Presentation Formats and Task Complexity on Online Consumers' Product Understanding , 2007, MIS Q..

[43]  Vicky Arnold,et al.  The Differential Use and Effect of Knowledge-Based System Explanations in Novice and Expert Judgement Decisions , 2006, MIS Q..

[44]  Jan Pries-Heje,et al.  Strategies for Design Science Research Evaluation , 2008, ECIS.

[45]  Shirley Gregor,et al.  Building theory in the sciences of the artificial , 2009, DESRIST.

[46]  Ritu Agarwal,et al.  Competing "Creatively" in Sponsored Search Markets: The Effect of Rank, Differentiation Strategy, and Competition on Performance , 2011, Inf. Syst. Res..

[47]  Vijay K. Vaishnavi,et al.  A Framework for Theory Development in Design Science Research: Multiple Perspectives , 2012, J. Assoc. Inf. Syst..

[48]  Anselm L. Strauss,et al.  Basics of qualitative research : techniques and procedures for developing grounded theory , 1998 .

[49]  Chuan-Hoo Tan,et al.  Addressing the Personalization-Privacy Paradox: An Empirical Assessment from a Field Experiment on Smartphone Users , 2013, MIS Q..

[50]  Izak Benbasat,et al.  Research Note: The Influence of Recommendations and Consumer Reviews on Evaluations of Websites , 2006, Inf. Syst. Res..

[51]  René Riedl,et al.  Computer Breakdown as a Stress Factor during Task Completion under Time Pressure: Identifying Gender Differences Based on Skin Conductance , 2013, Adv. Hum. Comput. Interact..

[52]  K. Perreault,et al.  Research Design: Qualitative, Quantitative, and Mixed Methods Approaches , 2011 .

[53]  Richard T. Watson,et al.  Analyzing the Past to Prepare for the Future: Writing a Literature Review , 2002, MIS Q..

[54]  Alan R. Hevner,et al.  Design Research in Information Systems , 2010 .

[55]  Jan Pries-Heje,et al.  Explanatory Design Theory , 2010, Bus. Inf. Syst. Eng..

[56]  Salvatore T. March,et al.  The Effects of State-Based and Event-Based Data Representation on User Performance in Query Formulation Tasks , 2006, MIS Q..

[57]  Shirley Gregor,et al.  The Nature of Theory in Information Systems , 2006, MIS Q..

[58]  Andrew M. Hardin,et al.  When Filling the Wait Makes it Feel Longer: A Paradigm Shift Perspective for Managing Online Delay , 2013, MIS Q..

[59]  Shawn P. Curley,et al.  Do Recommender Systems Manipulate Consumer Preferences? A Study of Anchoring Effects , 2013, Inf. Syst. Res..