Design Science Research: Evaluation in the Lens of Big Data Analytics

Given the different types of artifacts and their various evaluation methods, one of the main challenges faced by researchers in design science research (DSR) is choosing suitable and efficient methods during the artifact evaluation phase. With the emergence of big data analytics, data scientists conducting DSR are also challenged with identifying suitable evaluation mechanisms for their data products. Hence, this conceptual research paper is set out to address the following questions. Does big data analytics impact how evaluation in DSR is conducted? If so, does it lead to a new type of evaluation or a new genre of DSR? We conclude by arguing that big data analytics should influence how evaluation is conducted, but it does not lead to the creation of a new genre of design research.

[1]  Martin Bichler,et al.  Design science in information systems research , 2006, Wirtschaftsinf..

[2]  Shirley Gregor,et al.  The Anatomy of a Design Theory , 2007, J. Assoc. Inf. Syst..

[3]  Martin Bichler,et al.  Business Analytics and Data Science: Once Again? , 2017, Bus. Inf. Syst. Eng..

[4]  Richard Baskerville,et al.  What design science is not , 2008, Eur. J. Inf. Syst..

[5]  Giovanni Seni,et al.  Ensemble Methods in Data Mining: Improving Accuracy Through Combining Predictions , 2010, Ensemble Methods in Data Mining.

[6]  Alan R. Hevner,et al.  POSITIONING AND PRESENTING DESIGN SCIENCE RESEARCH FOR MAXIMUM IMPACT 1 , 2013 .

[7]  Salvatore T. March,et al.  Design and natural science research on information technology , 1995, Decis. Support Syst..

[8]  Björn Niehaves,et al.  Design science research genres: introduction to the special issue on exemplars and criteria for applicable design science research , 2018, Eur. J. Inf. Syst..

[9]  Terry Anthony Byrd,et al.  Big data analytics: Understanding its capabilities and potential benefits for healthcare organizations , 2018 .

[10]  Saul J. Berman,et al.  Embracing connectedness: insights from the IBM 2012 CEO study , 2013 .

[11]  Vasant Dhar,et al.  Data science and prediction , 2012, CACM.

[12]  A. Gunasekaran,et al.  Big data analytics in logistics and supply chain management: Certain investigations for research and applications , 2016 .

[13]  R. Boire.  Predictive analytics: The power to predict who will click, buy, lie, or die , 2013 .

[14]  Arun Rai,et al.  Editor's comments: diversity of Design Science Research , 2017 .

[15]  Alan R. Hevner,et al.  Design Science Research in Design Science Research in Information Systems , 2011 .

[16]  Daniel E. O'Leary,et al.  Embedding AI and Crowdsourcing in the Big Data Lake , 2014, IEEE Intelligent Systems.

[17]  R. Kitchin,et al.  Big Data, new epistemologies and paradigm shifts , 2014, Big Data Soc..

[18]  Michael D. Myers,et al.  A set of ethical principles for design science research in information systems , 2014, Inf. Manag..

[19]  Alan R. Hevner,et al.  Design Science Research Contributions: Finding a Balance between Artifact and Theory , 2018, J. Assoc. Inf. Syst..

[20]  Moutaz Haddara,et al.  BIG DATA ANALYTICS: A TEXT MINING-BASED LITERATURE ANALYSIS , 2014 .

[21]  Les Gasser,et al.  A Design Theory for Systems That Support Emergent Knowledge Processes , 2002, MIS Q..

[22]  Jan Pries-Heje,et al.  A Comprehensive Framework for Evaluation in Design Science Research 1 , 2022 .

[23]  Jan Pries-Heje,et al.  Strategies for Design Science Research Evaluation , 2008, ECIS.

[24]  Lorin M. Hitt,et al.  Strength in Numbers: How Does Data-Driven Decisionmaking Affect Firm Performance? , 2011, ICIS 2011.

[25]  Arun Rai,et al.  Editor's comments: avoiding type III errors: formulating IS research problems that matter , 2017 .

[26]  Ralf Klischewski,et al.  Theory-driven or process-driven prediction? Epistemological challenges of big data analytics , 2017, Journal of Big Data.

[27]  M. Hilbert,et al.  Big Data for Development: A Review of Promises and Challenges , 2016 .

[28]  Samir Chatterjee,et al.  Writing My next Design Science Research Master-piece: But How Do I Make a Theoretical Contribution to DSR? , 2015, ECIS.

[29]  Jung P. Shim,et al.  Big Data and Analytics: Issues, Solutions, and ROI , 2015, Commun. Assoc. Inf. Syst..

[30]  S. Chatterjee,et al.  Design Science Research in Information Systems , 2010 .