Research Perspectives: Design Theory Indeterminacy: What Is it, How Can it Be Reduced, and Why Did the Polar Bear Drown?

Design science research strives to be practical and relevant. Yet few researchers have examined the extent to which practitioners can meaningfully utilize theoretical knowledge produced by design science research in solving concrete real-world problems. Are design theories developed by scientists readily amenable to application by practitioners? Does the application of a theory by practitioners always lead to the outcomes predicted (by the scientists)? We examine a particularly difficult challenge—ensuring that the development and deployment of an IT artifact by practitioners based on a design theory result in appropriate changes in the environment predicted by the design theory. As we show in our paper, a gulf exists between theoretical propositions and concrete issues faced in practice—a challenge we refer to as design theory indeterminacy . Design theory indeterminacy might result in considerable ambiguity when implementing a design theory in practice and reduce the potential relevance of information systems knowledge. In this paper, we articulate the problem of design theory indeterminacy, examine factors that contribute to it, and suggest fruitful directions for future research to help reduce it.

[1]  A. Clements Citizen Science: Public Participation in Environmental Research , 2013 .

[2]  Jan vom Brocke,et al.  Digital Nudging , 2016, Business & Information Systems Engineering.

[3]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[4]  Shirley Gregor,et al.  Reflection, Abstraction And Theorizing In Design And Development Research , 2013, ECIS.

[5]  Ailene K. Ettinger,et al.  The science of citizen science: Exploring barriers to use as a primary research tool , 2017 .

[6]  Jingjing Li,et al.  TheoryOn: Designing a Construct-Based Search Engine to Reduce Information Overload for Behavioral Science Research , 2016, DESRIST.

[7]  J. G. Snodgrass,et al.  A standardized set of 260 pictures: norms for name agreement, image agreement, familiarity, and visual complexity. , 1980, Journal of experimental psychology. Human learning and memory.

[8]  Daniel L. Schacter,et al.  Priming effects in word-fragment completion are independent of recognition memory. , 1982 .

[9]  John R. Venable,et al.  Rethinking Design Theory in Information Systems , 2013, DESRIST.

[10]  Sandeep Purao,et al.  Truth or Dare: The Ontology Question in Design Science Research , 2013, J. Database Manag..

[11]  Shirley Gregor,et al.  Prescriptive Knowledge in IS Research: Conceptualizing Design Principles in Terms of Materiality, Action, and Boundary Conditions , 2015, 2015 48th Hawaii International Conference on System Sciences.

[12]  Roman Lukyanenko,et al.  Representing Crowd Knowledge: Guidelines for Conceptual Modeling of User-generated Content , 2017, J. Assoc. Inf. Syst..

[13]  Pierre-Majorique Léger,et al.  Smartphone withdrawal creates stress: A moderated mediation model of nomophobia, social threat, and phone withdrawal context , 2018, Comput. Hum. Behav..

[14]  Les Gasser,et al.  A Design Theory for Systems That Support Emergent Knowledge Processes , 2002, MIS Q..

[15]  S. Dasgupta Technology and Creativity , 2006, Woman with a Movie Camera.

[16]  Jordan Raddick,et al.  Galaxy Zoo: Morphological Classification and Citizen Science , 2011, 1104.5513.

[17]  Göran Goldkuhl,et al.  DESIGN THEORIES IN INFORMATION SYSTEMS - A NEED FOR MULTI-GROUNDING , 2004 .

[18]  T. J. Teorey,et al.  A logical design methodology for relational databases using the extended entity-relationship model , 1986, CSUR.

[19]  Wayne D. Gray,et al.  Basic objects in natural categories , 1976, Cognitive Psychology.

[20]  Roman Lukyanenko,et al.  The IQ of the Crowd: Understanding and Improving Information Quality in Structured User-Generated Content , 2014, Inf. Syst. Res..

[21]  Line Dubé,et al.  Rigor in Information Systems Positivist Case Research: Current Practices , 2003, MIS Q..

[22]  R. J. Bogumil,et al.  The reflective practitioner: How professionals think in action , 1985, Proceedings of the IEEE.

[23]  Roman Lukyanenko,et al.  Representing instances: the case for reengineering conceptual modelling grammars , 2018, Eur. J. Inf. Syst..

[24]  Tuure Tuunanen,et al.  Design Science Research Process: A Model for Producing and Presenting Information Systems Research , 2020, ArXiv.

[25]  Alan R. Hevner,et al.  Accumulation and Evolution of Design Knowledge in Design Science Research: A Journey Through Time and Space , 2020 .

[26]  Viswanath Venkatesh,et al.  Scholarly Commentaries on Hirschheim's "Against Theory" , 2019, J. Assoc. Inf. Syst..

[27]  Alan R. Hevner,et al.  Design Science Research Contributions: Finding a Balance between Artifact and Theory , 2018, J. Assoc. Inf. Syst..

[28]  Verena Tiefenbeck Fostering sustainable consumer decisions in practice: The impact of real-time feedback on resource consumption and voluntary carbon offsetting , 2016 .

[29]  Roman Lukyanenko,et al.  Citizen Science: An Information Quality Research Frontier , 2019, Information Systems Frontiers.

[30]  Thomas Hillman,et al.  Getting it Right or Being Top Rank: Games in Citizen Science , 2018 .

[31]  Monica Chiarini Tremblay,et al.  Basic Classes in Conceptual Modeling: Theory and Practical Guidelines , 2020, J. Assoc. Inf. Syst..

[32]  Bradley C. Love,et al.  Structural Priming as Structure-Mapping: Children Use Analogies From Previous Utterances to Guide Sentence Production , 2011, Cogn. Sci..

[33]  Alan R. Hevner,et al.  Utilizing, Producing, and Contributing Design Knowledge in DSR Projects , 2018, DESRIST.

[34]  Daniel R. Dolk,et al.  Design Principles for Virtual Worlds , 2011, MIS Q..

[35]  David P. Anderson,et al.  Scientists@Home: What Drives the Quantity and Quality of Online Citizen Science Participation? , 2014, PloS one.

[36]  Veda C. Storey,et al.  Reverse Engineering of Relational Databases: Extraction of an EER Model from a Relational Database , 1994, Data Knowl. Eng..

[37]  M. Goodchild Citizens as sensors: the world of volunteered geography , 2007 .

[38]  Leona Chandra Kruse,et al.  Tensions in design principle formulation and reuse , 2017 .

[39]  Ron Weber,et al.  On the ontological expressiveness of information systems analysis and design grammars , 1993, Inf. Syst. J..

[40]  Roman Lukyanenko,et al.  Easier citizen science is better , 2011, Nature.

[41]  Olga Sourina,et al.  Stability of Features in Real-Time EEG-based Emotion Recognition Algorithm , 2014, 2014 International Conference on Cyberworlds.

[42]  Maria-Eugenia Iacob,et al.  In Search of Paradigms : identifying the theoretical foundations of the information system field , 2010 .

[43]  Marta Indulska,et al.  Assessing Representation Theory with a Framework for Pursuing Success and Failure , 2017, MIS Q..

[44]  Allen S. Lee A Scientific Methodology for MIS Case Studies , 1989, MIS Q..

[45]  Roman Lukyanenko,et al.  Expecting the Unexpected: Effects of Data Collection Design Choices on the Quality of Crowdsourced User-Generated Content , 2019, MIS Q..

[46]  Hani Safadi,et al.  Social Media Affordances for Connective Action: An Examination of Microblogging Use During the Gulf of Mexico Oil Spill , 2017, MIS Q..

[47]  Ofer Arazy,et al.  A Theory-Driven Design Framework for Social Recommender Systems , 2010, J. Assoc. Inf. Syst..

[48]  Shirley Gregor,et al.  The Anatomy of a Design Theory , 2007, J. Assoc. Inf. Syst..

[49]  Christine Nadel,et al.  Case Study Research Design And Methods , 2016 .

[50]  Ron Kohavi,et al.  The Surprising Power of Online Experiments , 2017 .

[51]  Colin Potts,et al.  Design of Everyday Things , 1988 .

[52]  Howard B. Lee,et al.  Foundations of Behavioral Research , 1973 .

[53]  Ron Weber,et al.  Evaluating and Developing Theories in the Information Systems Discipline , 2012, J. Assoc. Inf. Syst..

[54]  J. Venable Incorporating design science research and critical research into an introductory business research methods course , 2011 .

[55]  C. P. Goodman,et al.  The Tacit Dimension , 2003 .

[56]  M. Bunge Finding philosophy in social science , 1996 .

[57]  Veda C. Storey,et al.  Inducing Creativity in Design Science Research , 2019, DESRIST.

[58]  Munir Mandviwalla,et al.  Generating and Justifying Design Theory , 2015, J. Assoc. Inf. Syst..

[59]  Yves Pigneur,et al.  A Design Theory for Visual Inquiry Tools , 2020, J. Assoc. Inf. Syst..

[60]  Donald A. Schön Educating the Reflective Practitioner: Toward a New Design for Teaching and Learning in the Professions , 1987 .

[61]  Alan R. Hevner,et al.  POSITIONING AND PRESENTING DESIGN SCIENCE RESEARCH FOR MAXIMUM IMPACT 1 , 2013 .

[62]  K. Weick What Theory Is Not, Theorizing Is , 1995 .

[63]  Juhani Iivari,et al.  A Paradigmatic Analysis of Information Systems As a Design Science , 2007, Scand. J. Inf. Syst..

[64]  Ron Weber,et al.  On the deep structure of information systems , 1995, Inf. Syst. J..

[65]  Joseph Feller,et al.  Procedurally Transparent Design Science Research: A Design Process Model , 2012, ICIS.

[66]  Ron Weber,et al.  Toward a Theory of the Deep Structure of Information Systems , 1990, ICIS.

[67]  Karin Ackermann,et al.  Categories and Concepts , 2003, Job 28. Cognition in Context.

[68]  G. Murphy,et al.  The Big Book of Concepts , 2002 .

[69]  Paulo B. Góes,et al.  Editor's comments: design science research in top information systems journals , 2014 .

[70]  A. Kruglanski,et al.  Features of Multifinality , 2013, Personality and social psychology review : an official journal of the Society for Personality and Social Psychology, Inc.

[71]  Stefan Seidel,et al.  Leaving the Beaten Tracks in Creative Work - A Design Theory for Systems that Support Convergent and Divergent Thinking , 2011, J. Assoc. Inf. Syst..

[72]  R. Bonney,et al.  Next Steps for Citizen Science , 2014, Science.

[73]  II ON SOCIOLOGICAL THEORIES OF THE MIDDLE RANGE , 2022 .

[74]  Jacky Akoka,et al.  A Taxonomy of Evaluation Methods for Information Systems Artifacts , 2015, J. Manag. Inf. Syst..

[75]  Roman Lukyanenko,et al.  Emerging problems of data quality in citizen science , 2016, Conservation biology : the journal of the Society for Conservation Biology.

[76]  Veda C. Storey,et al.  Aesthetics in design science research , 2018, Eur. J. Inf. Syst..

[77]  Jacky Akoka,et al.  Artifact Evaluation in Information Systems Design-Science Research - a Holistic View , 2014, PACIS.

[78]  Omar A. El Sawy,et al.  Assessing Information System Design Theory in Perspective: How Useful Was our 1992 Initial Rendition? , 2004 .

[79]  Donald A. Norman,et al.  Affordance, conventions, and design , 1999, INTR.

[80]  Daniel G. Goldstein,et al.  Beyond nudges: Tools of a choice architecture , 2012 .

[81]  Kevin Crowston,et al.  Surveying the citizen science landscape , 2014, First Monday.

[82]  Paul M. Leonardi,et al.  When Flexible Routines Meet Flexible Technologies: Affordance, Constraint, and the Imbrication of Human and Material Agencies , 2011, MIS Q..

[83]  Sandeep Purao,et al.  Making Use of Design Principles , 2016, DESRIST.

[84]  Vijay K. Vaishnavi,et al.  A Framework for Theory Development in Design Science Research: Multiple Perspectives , 2012, J. Assoc. Inf. Syst..

[85]  Elgar Fleisch,et al.  Real-time feedback reduces energy consumption among the broader public without financial incentives , 2019, Nature Energy.

[86]  Hubert L. Dreyfus,et al.  What computers still can't do - a critique of artificial reason , 1992 .

[87]  Daniel L. Moody,et al.  The “Physics” of Notations: Toward a Scientific Basis for Constructing Visual Notations in Software Engineering , 2009, IEEE Transactions on Software Engineering.

[88]  Roman Lukyanenko,et al.  Participatory Design for User-generated Content: Understanding the challenges and moving forward , 2016, Scand. J. Inf. Syst..

[89]  Verena Tiefenbeck,et al.  Real-world impact of Information Systems: The effect of seemingly small design choices , 2017 .

[90]  M. Bradley,et al.  The International Affective Picture System (IAPS) in the study of emotion and attention. , 2007 .

[91]  Dan Woods,et al.  APIs - a Strategy Guide: Creating Channels with Application Programming Interfaces , 2012 .

[92]  John R. Venable,et al.  The role of theory and theorising in Design Science research , 2006 .

[93]  Namir,et al.  Authors , 1947, Praxis der Kinderpsychologie und Kinderpsychiatrie.

[94]  Eva J. Lewandowski,et al.  Influence of volunteer and project characteristics on data quality of biological surveys , 2015, Conservation biology : the journal of the Society for Conservation Biology.

[95]  Elgar Fleisch,et al.  Overcoming Salience Bias: How Real-Time Feedback Fosters Resource Conservation , 2016, Manag. Sci..

[96]  R. Sieber Public Participation Geographic Information Systems: A Literature Review and Framework , 2006 .

[97]  Matt Germonprez,et al.  The Potential for Citizen Science in Information Systems Research , 2017, Commun. Assoc. Inf. Syst..

[98]  Arun Rai,et al.  Editor's comments: avoiding type III errors: formulating IS research problems that matter , 2017 .

[99]  Maha Shaikh,et al.  Code Forking and Software Development Project Sustainability: Evidence from GitHub , 2019, ICIS.

[100]  Roman Lukyanenko,et al.  Guidelines for Establishing Instantiation Validity in IT Artifacts: A Survey of IS Research , 2015, DESRIST.

[101]  Shirley Gregor,et al.  The Nature of Theory in Information Systems , 2006, MIS Q..

[102]  Matt Germonprez,et al.  Secondary Design: A Case of Behavioral Design Science Research , 2011, J. Assoc. Inf. Syst..

[103]  Neville A Stanton,et al.  The Design with Intent Method: a design tool for influencing user behaviour. , 2010, Applied ergonomics.

[104]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[105]  V. Tiefenbeck,et al.  Bring behaviour into the digital transformation , 2017, Nature Energy.

[106]  Vijay K. Vaishnavi,et al.  Theory Development in Design Science Research: Anatomy of a Research Project , 2008 .

[107]  Matt Germonprez,et al.  A Theory of Tailorable Technology Design , 2007, J. Assoc. Inf. Syst..

[108]  Edoardo Lozza,et al.  The Ethics of Influence. Government in the Age of Behavioral Science , 2017 .

[109]  Andrea Resca,et al.  A design theory for digital platforms supporting online communities: a multiple case study , 2015, J. Inf. Technol..

[110]  P. Wesley Schultz,et al.  Personalized Normative Feedback and the Moderating Role of Personal Norms , 2016 .

[111]  Paul Johannesson,et al.  The case for classes and instances - a response to representing instances: the case for reengineering conceptual modelling grammars , 2019, Eur. J. Inf. Syst..

[112]  Veda C. Storey,et al.  Genres of Inquiry in Design-Science Research: Justification and Evaluation of Knowledge Production , 2015, MIS Q..

[113]  M. Heidegger,et al.  Being and time : a translation of Sein und Zeit , 1996 .

[114]  Omar El Sawy,et al.  Building an Information System Design Theory for Vigilant EIS , 1992, Inf. Syst. Res..

[115]  Shirley Gregor,et al.  Causality: the elephant in the room in information systems epistemology , 2011, ECIS.

[116]  Rudy Hirschheim,et al.  Against Theory: With Apologies to Feyerabend , 2019, J. Assoc. Inf. Syst..

[117]  Pamela Baxter,et al.  Qualitative Case Study Methodology: Study Design and Implementation for Novice Researchers , 2008 .

[118]  Alan R. Hevner,et al.  Design Research in Information Systems: Theory and Practice , 2010 .

[119]  Jeffrey Parsons,et al.  Is Query Reuse Potentially Harmful? Anchoring and Adjustment in Adapting Existing Database Queries , 2010, Inf. Syst. Res..

[120]  Yair Wand,et al.  Using Cognitive Principles to Guide Classification in Information Systems Modeling , 2008, MIS Q..

[121]  K. Weick Theory Construction as Disciplined Imagination , 1989 .

[122]  Roman Lukyanenko,et al.  Instantiation Validity in IS Design Research , 2014, DESRIST.

[123]  Shirley Gregor,et al.  Untangling causality in design science theorizing , 2010 .

[124]  José Antonio Méndez Sanz,et al.  Thinking through technology: The path between engineering and philosophy , 1995 .