Exploring the Scientific Impact of Information Systems Design Science Research: A Scientometric Study

Design science is a fundamental research stream that contends its position in the information systems discipline. While ongoing debates address the relative importance of design science contributions in the information systems community, insights into the scientific impact of design science research (DSR) are missing and this lack of understanding arguably poses challenges to an informed discourse. To identify the most influential papers and those factors that explain their scientific impact, this paper presents an exploratory study of the scientific impact of DSR papers published in the AIS Senior Scholars' Basket of Journals. We uncover the current DSR landscape by taking stock of influential papers and theories and develop a model to explain the scientific impact of DSR papers. Our findings show that scientific impact is significantly explained by theorization and novelty. We discuss how the implications of our work can be projected on the overarching discourse on DSR.

[1]  Johann Füller,et al.  Co-Creation in Virtual Worlds: The Design of the User Experience , 2011, MIS Q..

[2]  Andrea Bergmann,et al.  Citation Indexing Its Theory And Application In Science Technology And Humanities , 2016 .

[3]  Rafael A. Gonzalez,et al.  Quo Vadis, Design Science? - A Survey of Literature , 2010, DESRIST.

[4]  W. Shadish,et al.  Author Judgements about Works They Cite: Three Studies from Psychology Journals , 1995 .

[5]  Ahmad Alturki,et al.  A Design Science Research Roadmap , 2011, PACIS.

[6]  Lutz Bornmann,et al.  A multilevel meta-analysis of studies reporting correlations between the h index and 37 different h index variants , 2011, J. Informetrics.

[7]  Jan Muntermann,et al.  A method for taxonomy development and its application in information systems , 2013, Eur. J. Inf. Syst..

[8]  Gwendolyn L. Kolfschoten,et al.  A Design Approach for Collaboration Processes: A Multimethod Design Science Study in Collaboration Engineering , 2009, J. Manag. Inf. Syst..

[9]  Shirley Gregor,et al.  The Nature of Theory in Information Systems , 2006, MIS Q..

[10]  Alexander Maedche,et al.  Anatomy of Knowledge Bases Used in Design Science Research - A Literature Review , 2012, DESRIST.

[11]  G. Gilbert.  Referencing as Persuasion , 1977 .

[12]  Paulo B. Góes,et al.  Editor's comments: commonalities across IS silos and intradisciplinary information systems research , 2013 .

[13]  R. Merton,et al.  The Sociology of Science: Theoretical and Empirical Investigations , 1975, Journal for the Scientific Study of Religion.

[14]  Arun Rai,et al.  Editor's comments: diversity of Design Science Research , 2017 .

[15]  David Arnott,et al.  Cognitive biases and decision support systems development: a design science approach , 2006, Inf. Syst. J..

[16]  Graham Pervan,et al.  Design Science in Decision Support Systems Research: An Assessment using the Hevner, March, Park, and Ram Guidelines , 2012, J. Assoc. Inf. Syst..

[17]  Ken Peffers,et al.  Collaboration and Author Order: Changing Patterns in IS Research , 2003, Commun. Assoc. Inf. Syst..

[18]  John Krogstie,et al.  Process models representing knowledge for action: a revised quality framework , 2006, Eur. J. Inf. Syst..

[19]  Roman Lukyanenko,et al.  Reconciling Theories with Design Choices in Design Science Research , 2013, DESRIST.

[20]  Alan R. Hevner,et al.  POSITIONING AND PRESENTING DESIGN SCIENCE RESEARCH FOR MAXIMUM IMPACT 1 , 2013 .

[21]  Richard Baskerville,et al.  Eating Our Own Cooking: Toward a More Rigorous Design Science of Research Methods , 2012 .

[22]  Mark A. Fuller,et al.  Research Standards for Promotion and Tenure in Information Systems , 2006, MIS Q..

[23]  Vijayan Sugumaran,et al.  The State of Design Science Research within the BISE Community: An Empirical Investigation , 2014, ICIS.

[24]  Richard Baskerville,et al.  A Design Theory for Secure Information Systems Design Methods , 2006, J. Assoc. Inf. Syst..

[25]  Samir Chatterjee,et al.  Meta-analysis of Design Science Research within the IS Community: Trends, Patterns, and Outcomes , 2010, DESRIST.

[26]  Peter Mertens,et al.  Memorandum on design-oriented information systems research , 2011, Eur. J. Inf. Syst..

[27]  Jennifer Jie Xu,et al.  The Social Identity of IS: Analyzing the Collaboration Network of the ICIS Conferences (1980-2005) , 2006, ICIS.

[28]  E Garfield,et al.  "Science Citation Index"--A New Dimension in Indexing. , 1964, Science.

[29]  Ron Weber,et al.  Still desperately seeking the IT artifact , 2003 .

[30]  P BostromRobert,et al.  MIS Problems and failures a socio-technical perspective part II , 1977 .

[31]  Claudia Löbbecke,et al.  Engaging scientometrics in Information Systems: Combining the Quantitative and Qualitative , 2010, ICIS.

[32]  Michael Wade,et al.  The Value of Scientometric Studies: An Introduction to a Debate on IS as a Reference Discipline , 2006, J. Assoc. Inf. Syst..

[33]  Richard Baskerville,et al.  What design science is not , 2008, Eur. J. Inf. Syst..

[34]  Per Ottar Seglen,et al.  Causal relationship between article citedness and journal impact , 1994 .

[35]  Yair Wand,et al.  Extending Classification Principles from Information Modeling to Other Disciplines , 2013, J. Assoc. Inf. Syst..

[36]  Duane P. Truex,et al.  Assessing Scholarly Influence: Using the Hirsch Indices to Reframe the Discourse , 2009, J. Assoc. Inf. Syst..

[37]  Yunjie Calvin Xu,et al.  Alternate Strategies for a Win-Win Seeking Agent in Agent-Human Negotiations , 2012, J. Manag. Inf. Syst..

[38]  Anthony F. J. van Raan,et al.  On Determinants of Citation Scores: A Case Study in Chemical Engineering , 1994, J. Am. Soc. Inf. Sci..

[39]  Michael H. MacRoberts,et al.  Problems of citation analysis: A critical review , 1989, JASIS.

[40]  George R. Widmeyer,et al.  Towards a Formal Approach to Information Systems Design Theory Using Category Theory , 2012, DESRIST.

[41]  Mathias Ekstedt,et al.  Using enterprise architecture analysis and interview data to estimate service response time , 2013, J. Strateg. Inf. Syst..

[42]  Shirley Gregor,et al.  The Anatomy of a Design Theory , 2007, J. Assoc. Inf. Syst..

[43]  Yair Wand,et al.  Using Cognitive Principles to Guide Classification in Information Systems Modeling , 2008, MIS Q..

[44]  E. Burton Swanson,et al.  A Simple Research Impacts Model Applied to the Information Systems Field , 2014, Commun. Assoc. Inf. Syst..

[45]  Ofer Arazy,et al.  A Theory-Driven Design Framework for Social Recommender Systems , 2010, J. Assoc. Inf. Syst..

[46]  William H. Starbuck,et al.  How Much Better are the Most Prestigious Journals? The Statistics of Academic Publication , 2005, Organ. Sci..

[47]  Vijay K. Vaishnavi,et al.  Theory Development in Design Science Research: Anatomy of a Research Project , 2008 .

[48]  Veda C. Storey,et al.  Genres of Inquiry in Design-Science Research: Justification and Evaluation of Knowledge Production , 2015, MIS Q..

[49]  Hsinchun Chen,et al.  CyberGate: A Design Framework and System for Text Analysis of Computer-Mediated Communication , 2008, MIS Q..

[50]  Daniel Robey,et al.  Identity, Legitimacy and the Dominant Research Paradigm: An Alternative Prescription for the IS Discipline: A Response to Benbasat and Zmud's Call for Returning to the IT Artifact , 2003, J. Assoc. Inf. Syst..

[51]  J. A. Stewart,et al.  Achievement and Ascriptive Processes in the Recognition of Scientific Articles , 1983 .

[52]  Jacky Akoka,et al.  A Taxonomy of Evaluation Methods for Information Systems Artifacts , 2015, J. Manag. Inf. Syst..

[53]  Omar A. El Sawy,et al.  Assessing Information System Design Theory in Perspective: How Useful Was our 1992 Initial Rendition? , 2004 .

[54]  Varun Grover,et al.  The Effect of an IS Article's Structure on Its Impact , 2010, Commun. Assoc. Inf. Syst..

[55]  Robert P. Bostrom,et al.  MIS Problems and failures: a sociotechnical perspective part I: the cause , 1977 .

[56]  Marten Schönherr,et al.  Artifact Types in Information Systems Design Science - A Literature Review , 2010, DESRIST.

[57]  Alan R. Hevner,et al.  Design Science in Information Systems Research , 2004, MIS Q..

[58]  S. Baldi.  Normative versus social constructivist processes in the allocation of citations : A network-analytic model , 1998 .

[59]  Ron Weber,et al.  Guidelines for Empirical Evaluations of Conceptual Modeling Grammars , 2009, J. Assoc. Inf. Syst..

[60]  Fred Niederman,et al.  Design science and the accumulation of knowledge in the information systems discipline , 2012, TMIS.

[61]  Paulo B. Góes,et al.  Editor's comments: inflection point: looking back or looking forward? , 2015 .

[62]  S. Chatterjee,et al.  Design Science Research in Information Systems , 2010 .

[63]  Joseph Feller,et al.  Procedurally Transparent Design Science Research: A Design Process Model , 2012, ICIS.

[64]  Konrad Paul Kording,et al.  Future impact: Predicting scientific success , 2012, Nature.

[65]  Detmar W. Straub,et al.  Knowledge Contributions in Design Science Research: A Meta-Analysis , 2014, DESRIST.

[66]  Stefan Seidel,et al.  Leaving the Beaten Tracks in Creative Work - A Design Theory for Systems that Support Convergent and Divergent Thinking , 2011, J. Assoc. Inf. Syst..

[67]  Jay F. Nunamaker,et al.  Creating High-Value Real-World Impact through Systematic Programs of Research , 2017, MIS Q..

[68]  Juhani Iivari,et al.  A Paradigmatic Analysis of Information Systems As a Design Science , 2007, Scand. J. Inf. Syst..

[69]  Jacky Akoka,et al.  The Structure of DSR Knowledge as Reflected by DESRIST - A Citation Analysis (2009-2015) , 2016, DESRIST.

[70]  F. E..  A Relational Model of Data Large Shared Data Banks , 2000 .

[71]  Kalle Lyytinen,et al.  Design theory for dynamic complexity in information infrastructures: the case of building internet , 2010, J. Inf. Technol..

[72]  Samir Chatterjee,et al.  A Design Science Research Methodology for Information Systems Research , 2008 .

[73]  Björn Niehaves,et al.  On the Relationship between the IT Artifact and Design Theory: The Case of Virtual Social Facilitation , 2012, DESRIST.

[74]  S. Ghoshal,et al.  Social Capital, Intellectual Capital, and the Organizational Advantage , 1998 .

[75]  Rajendra P. Srivastava,et al.  An Information Systems Security Risk Assessment Model Under the Dempster-Shafer Theory of Belief Functions , 2006, J. Manag. Inf. Syst..

[76]  M. Moravcsik,et al.  Some Results on the Function and Quality of Citations , 1975 .

[77]  Shirley Gregor,et al.  Building theory in the sciences of the artificial , 2009, DESRIST.

[78]  Izak Benbasat,et al.  The Identity Crisis Within the IS Discipline: Defining and Communicating the Discipline's Core Properties , 2003, MIS Q..

[79]  Christian Fischer,et al.  The Information Systems Design Science Research Body Of Knowledge - A Citation Analysis In Recent Top-Journal Publications , 2011, PACIS.

[80]  Alan R. Hevner,et al.  A Three Cycle View of Design Science Research , 2007 .

[81]  Robert D. Galliers,et al.  Change as Crisis or Growth? Toward a Trans-disciplinary View of Information Systems as a Field of Study: A Response to Benbasat and Zmud's Call for Returning to the IT Artifact , 2003, J. Assoc. Inf. Syst..

[82]  Jan Marco Leimeister,et al.  Design, Implementation, and Evaluation of Trust-Supporting Components in Virtual Communities for Patients , 2005, J. Manag. Inf. Syst..

[83]  Alan M. Davis,et al.  A Unified Model of Requirements Elicitation , 2004, J. Manag. Inf. Syst..

[84]  Matt Germonprez,et al.  A Theory of Tailorable Technology Design , 2007, J. Assoc. Inf. Syst..

[85]  Claudia Löbbecke,et al.  Research Importance in the Information Systems Field: A Citations Analysis , 2007, ICIS.

[86]  Kevin Crowston,et al.  Assessing IS Research Impact , 2015, Commun. Assoc. Inf. Syst..

[87]  Varun Grover,et al.  What Affects Citation Counts in MIS Research Articles? An Empirical Investigation , 2014, Commun. Assoc. Inf. Syst..

[88]  Juha-Pekka Tolvanen,et al.  Managing Evolutionary Method Engineering by Method Rationale , 2004, J. Assoc. Inf. Syst..

[89]  Olivia R. Liu Sheng,et al.  Formulating the Data-Flow Perspective for Business Process Management , 2006, Inf. Syst. Res..

[90]  Jennifer Jie Xu,et al.  Business Intelligence in Blogs: Understanding Consumer Interactions and Communities , 2012, MIS Q..

[91]  E. Garfield.  The history and meaning of the journal impact factor. , 2006, JAMA.

[92]  R. Kostoff.  The (scientific) wealth of nations , 2004 .

[93]  Alok Gupta,et al.  GIST: A Model for Design and Management of Content and Interactivity of Customer-Centric Web Sites , 2004, MIS Q..

[94]  Ahmad Alturki,et al.  Information System Design Theory: A Lifecycle Perspective , 2016, DESRIST.

[95]  Omar El Sawy,et al.  Building an Information System Design Theory for Vigilant EIS , 1992, Inf. Syst. Res..

[96]  Anna Sidorova,et al.  Uncovering the Intellectual Core of the Information Systems Discipline , 2008, MIS Q..

[97]  Vijay K. Vaishnavi,et al.  A Framework for Theory Development in Design Science Research: Multiple Perspectives , 2012, J. Assoc. Inf. Syst..

[98]  Erik Brynjolfsson,et al.  Network Analysis for Predicting Academic Impact , 2013, ICIS.

[99]  Rudy Hirschheim,et al.  Crisis in the IS Field? A Critical Reflection on the State of the Discipline , 2003, J. Assoc. Inf. Syst..

[100]  T. Judge,et al.  What Causes a Management Article to be Cited—Article, Author, or Journal? , 2007 .

[101]  Detmar W. Straub,et al.  Editor's comments: why top journals accept your paper , 2009 .

[102]  Alan R. Hevner,et al.  A Fitness-Utility Model for Design Science Research , 2011, TMIS.

[103]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[104]  Kalle Lyytinen,et al.  The Legacy of "Power and Politics" in Disciplinary Discourse: A Citation Analysis , 2006, ICIS.

[105]  Gerardine DeSanctis,et al.  The Social Life of Information Systems Research: A Response to Benbasat and Zmud's Call for Returning to the IT Artifact , 2003, J. Assoc. Inf. Syst..

[106]  Lutz Bornmann,et al.  What do citation counts measure? A review of studies on citing behavior , 2008, J. Documentation.

[107]  Jeffrey G. Gray,et al.  DSMDiff: a differentiation tool for domain-specific models , 2007 .

[108]  Carol Tenopir,et al.  The role of trust and authority in the citation behaviour of researchers , 2015, Inf. Res..

[109]  Loet Leydesdorff,et al.  Towards a theory of citation? , 1987, Scientometrics.

[110]  W. Myers,et al.  Atypical Combinations and Scientific Impact , 2013 .

[111]  John Mingers,et al.  Exploring the dynamics of journal citations: Modelling with s-curves , 2008, J. Oper. Res. Soc..

[112]  Andreas Drechsler,et al.  What Kinds of Artifacts Are We Designing? An Analysis of Artifact Types and Artifact Relevance in IS Journal Publications , 2014, DESRIST.

[113]  K. Knorr-Cetina.  The Manufacture of Knowledge: an Essay on the Constructivist and Contextual Nature of Science , 1985 .

[114]  Jan Pries-Heje,et al.  Explanatory Design Theory , 2010, Bus. Inf. Syst. Eng..

[115]  Dongsong Zhang,et al.  The Effects of Tree-View Based Presentation Adaptation on Model Web Browsing , 2011, MIS Q..

[116]  Robert J. Kauffman,et al.  Making Sense of Technology Trends in the Information Technology Landscape: A Design Science Approach , 2008, MIS Q..

[117]  Henry C. Lucas,et al.  The information systems identity crisis , 2005 .

[118]  Jan Pries-Heje,et al.  Theorizing in Design Science Research , 2011, DESRIST.

[119]  Xin Luo,et al.  A Scientometric Investigation into the Validity of IS Journal Quality Measures , 2007, J. Assoc. Inf. Syst..

[120]  I. Lakatos.  Falsification and the Methodology of Scientific Research Programmes , 1976 .

[121]  Kai Zimmermann,et al.  Read This! How to Boost the Interest towards Research Articles - A Study on SSRN Research Impact , 2014, ICIS.

[122]  Salvatore T. March,et al.  Design and natural science research on information technology , 1995, Decis. Support Syst..

[123]  John R. Venable,et al.  Rethinking Design Theory in Information Systems , 2013, DESRIST.

[124]  Weiguo Fan,et al.  Genetic Programming-Based Discovery of Ranking Functions for Effective Web Search , 2005, J. Manag. Inf. Syst..

[125]  Jan Pries-Heje,et al.  The Design Theory Nexus , 2008, MIS Q..

[126]  Kène Henkens,et al.  What Makes a Scientific article Influential , 2000 .

[127]  Fang Xu,et al.  The drivers of citations in management science journals , 2010, Eur. J. Oper. Res..

[128]  Alexander Benlian,et al.  Factors Affecting the Scientific Impact of Literature Reviews: A Scientometric Study , 2016, ICIS.