Engaging scientometrics in Information Systems: Combining the Quantitative and Qualitative

The goal of this study is to stimulate a broader and deeper application of scientometric theory in information systems (IS) research, specifically to emphasize the benefits of scientometrics in the conceptual development of the IS field. The study reviews the underlying theories of scientometrics, specifically how they combine both quantitative and qualitative approaches towards research evaluation. Based on the analysis of selected scientometric studies in IS, this study finds significant room for future research in the virtually untouched landscape of scientometric theories.

[1]  K. Clark,et al.  America's Psychologists: A Survey Of A Growing Profession , 2012 .

[2]  E. Schiebel,et al.  Identification and characterisation of technological topics in the field of Molecular Biology , 2009, Scientometrics.

[3]  Duane P. Truex,et al.  Assessing Scholarly Influence: Using the Hirsch Indices to Reframe the Discourse , 2009, J. Assoc. Inf. Syst..

[4]  Clyde W. Holsapple,et al.  A new map for knowledge dissemination channels , 2009, CACM.

[5]  Y. Reich.  What is a reference? , 2008, Research in Engineering Design.

[6]  Leslie P. Willcocks,et al.  The ranking of top IS journals: a perspective from the London School of Economics , 2008, Eur. J. Inf. Syst..

[7]  Mohamed Khalifa,et al.  Demographic changes in IS research productivity and impact , 2008, CACM.

[8]  Lutz Bornmann,et al.  What do citation counts measure? A review of studies on citing behavior , 2008, J. Documentation.

[9]  Clyde W. Holsapple,et al.  A publication power approach for identifying premier information systems journals , 2008, J. Assoc. Inf. Sci. Technol..

[10]  Carola Schauer.  Relevance and success of IS teaching and research: An analysis of the "Relevance Debate" , 2018 .

[11]  Michael White,et al.  Determinants of faculty research productivity in information systems: An empirical analysis of the impact of academic origin and academic affiliation , 2009, Scientometrics.

[12]  Xin Luo,et al.  A Scientometric Investigation into the Validity of IS Journal Quality Measures , 2007, J. Assoc. Inf. Syst..

[13]  Robert D. Galliers,et al.  An alternative perspective on citation classics: Evidence from the first 10 years of the European Conference on Information Systems , 2007, Inf. Manag..

[14]  Raquel Benbunan-Fich,et al.  Analyzing IS research productivity: an inclusive approach to global IS scholarship , 2007, Eur. J. Inf. Syst..

[15]  Markus,et al.  Editor's Comments: Looking for a Few Good Concepts...and Theories...for the Information Systems Field , 2007 .

[16]  Claudia Löbbecke,et al.  Research Importance in the Information Systems Field: A Citations Analysis , 2007, ICIS.

[17]  Tom Groot,et al.  Research quality and efficiency: An analysis of assessments and management issues in Dutch economics and business research programs , 2006 .

[18]  Shirley Gregor,et al.  The Nature of Theory in Information Systems , 2006, MIS Q..

[19]  Hee-Woong Kim,et al.  Information systems citation patterns from International Conference on Information Systems articles , 2006, J. Assoc. Inf. Sci. Technol..

[20]  Henry M. Kim,et al.  Information Systems is Not a Reference Discipline (And What We Can Do About It) , 2006, J. Assoc. Inf. Syst..

[21]  Henry M. Kim,et al.  If the Tree of IS Knowledge Falls in a Forest, Will Anyone Hear?: A Commentary on Grover et al , 2006, J. Assoc. Inf. Syst..

[22]  Varun Grover,et al.  About Reference Disciplines and Reference Differences: A Critique of Wade et al , 2006, J. Assoc. Inf. Syst..

[23]  Varun Grover,et al.  A Citation Analysis of the Evolution and State of Information Systems within a Constellation of Reference Disciplines , 2006, J. Assoc. Inf. Syst..

[24]  Detmar W. Straub,et al.  The Value of Scientometric Studies: An Introduction to a Debate on IS as a Reference Discipline , 2006, J. Assoc. Inf. Syst..

[25]  Pairin Katerattanakul,et al.  Is information systems a reference discipline? , 2006, CACM.

[26]  Mark A. Fuller,et al.  Research Standards for Promotion and Tenure in Information Systems , 2006, MIS Q..

[27]  K. Lyytinen,et al.  Information systems : the state of the field , 2006 .

[28]  Richard T. Watson,et al.  Beyond Rigor and Relevance Towards Responsibility and Reverberation: Information Systems Research That Really Matters , 2006, Commun. Assoc. Inf. Syst..

[29]  Blaise Cronin,et al.  Metatheorizing citation , 2006, Scientometrics.

[30]  Kalle Lyytinen,et al.  The Legacy of "Power and Politics" in Disciplinary Discourse: A Citation Analysis , 2006, ICIS.

[31]  Eugene Garfield,et al.  Can Nobel Prize Winners Be Predicted? , 2006 .

[32]  Riyaz Sikora,et al.  Assessing the relative influence of journals in a citation network , 2005, CACM.

[33]  Brian D. Cameron,et al.  Trends in the Usage of ISI Bibliometric Data: Uses, Abuses, and Implications , 2005 .

[34]  Mark D. Miller,et al.  Examining differences across journal rankings , 2005, CACM.

[35]  Susan E. Cozzens,et al.  What do citations count? the rhetoric-first model , 1989, Scientometrics.

[36]  Loet Leydesdorff,et al.  Towards a theory of citation? , 1987, Scientometrics.

[37]  Henry G. Small,et al.  Citation context analysis of a co-citation cluster: Recombinant-DNA , 1980, Scientometrics.

[38]  Derek J. de Solla Price,et al.  Editorial statements , 2005, Scientometrics.

[39]  Stuart J. Barnes,et al.  Assessing the value of IS journals , 2005, CACM.

[40]  Jack Shih-Chieh Hsu,et al.  An Evaluation of Publication Productivity in Information Systems: 1999 to 2003 , 2005, Commun. Assoc. Inf. Syst..

[41]  Henry G. Small,et al.  On the shoulders of Robert Merton: Towards a normative theory of citation , 2004, Scientometrics.

[42]  Paul Benjamin Lowry,et al.  Global Journal Prestige and Supporting Disciplines: A Scientometric Study of Information Systems Journals , 2004, J. Assoc. Inf. Syst..

[43]  Irina Marshakova-Shaikevich,et al.  Scientometric Perspectives of the Analysis of Chemical Terminology , 2001, Scientometrics.

[44]  Bonnie Mae Kaplan,et al.  Information systems research: Relevant theory and informed practice , 2004 .

[45]  Concepción S. Wilson,et al.  Social network analysis and citation network analysis: Complementary approaches to the study of scientific communication. Sponsored by SIG MET , 2005, ASIST.

[46]  Soongoo Hong,et al.  Objective quality ranking of computing journals , 2003, CACM.

[47]  Izak Benbasat,et al.  The Identity Crisis Within the IS Discipline: Defining and Communicating the Discipline's Core Properties , 2003, MIS Q..

[48]  Pairin Katerattanakul,et al.  Are European IS Journals under-rated? An answer based on citation analysis , 2003, Eur. J. Inf. Syst..

[49]  K. Peffers,et al.  Identifying and Evaluating the Universe of Outlets for Information Systems Research: Ranking the Journals , 2003 .

[50]  Paul Gray,et al.  Introduction to the Debate on the Core of the Information Systems Field , 2003, Commun. Assoc. Inf. Syst..

[51]  Penny O’Connor,et al.  American Society for Information Science and Technology, Annual Conference , 2002 .

[52]  Detmar W. Straub,et al.  Measuring Researcher-Production in Information Systems , 2002, J. Assoc. Inf. Syst..

[53]  Bill C. Hardgrave,et al.  Forums for information systems scholars: III , 2001, Inf. Manag..

[54]  Nikolaos A. Mylonopoulos,et al.  Global perceptions of IS journals , 2001 .

[55]  Lori N. K. Leonard,et al.  Citation classics from the information systems literature , 2000, Inf. Manag..

[56]  van Raan,et al.  R&D evaluation at the beginning of the new century , 2000 .

[57]  Blaise Cronin,et al.  Semiotics and evaluative bibliometrics , 2000, J. Documentation.

[58]  Reyes Gonzalez,et al.  An analysis of research in information systems (1981-1997) , 2000, Inf. Manag..

[59]  Susan Athey,et al.  An Evaluation of Research Productivity in Academic IT , 2000, Commun. Assoc. Inf. Syst..

[60]  E. Haigh.  Science in Russia and the Soviet Union , 2000 .

[61]  J. Wyatt.  Decision support systems. , 2000, Journal of the Royal Society of Medicine.

[62]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[63]  Il Im,et al.  Topics of interest in IS: evolution of themes and differences between research and practice , 1999, Inf. Manag..

[64]  Donald H. Drury,et al.  A Historiographical Examination of Information Systems , 1999, Commun. Assoc. Inf. Syst..

[65]  Thomas H. Davenport,et al.  Rigor vs. relevance revisited: response to Benbasat and Zmud , 1999 .

[66]  Izak Benbasat,et al.  Empirical Research in Information Systems: The Practice of Relevance , 1999, MIS Q..

[67]  Anthony R. Hendrickson,et al.  Research Commentary. Academic Rewards for Teaching, Research, and Service: Data and Discourse , 1999, Inf. Syst. Res..

[68]  P. Wouters.  The citation culture , 1999 .

[69]  Steven Walczak,et al.  A Re-Evaluation of Information Systems Publication Forums , 1999 .

[70]  M. Lynne Markus,et al.  Beyond Rigor and Relevance: Producing Consumable Research about Information Systems , 1998 .

[71]  Eugenegarfield.  Citation Indexes in Sociological and Historical Research , 1998 .

[72]  Kee Young Kim,et al.  An Assessment of Individual and Institutional Research Productivity in DEA , 1998 .

[73]  Bill C. Hardgrave,et al.  Forums for MIS scholars , 1997, CACM.

[74]  Sean B. Eom,et al.  The Contributions of Organizational Science to the Development of Decision Support Systems Research Subspecialties , 1996, J. Am. Soc. Inf. Sci..

[75]  Daniel Robey,et al.  Research Commentary: Diversity in Information Systems Research: Threat, Promise, and Responsibility , 1996, Inf. Syst. Res..

[76]  Izak Benbasat,et al.  Research Commentary: Rethinking "Diversity" in Information Systems Research , 1996, Inf. Syst. Res..

[77]  Sean B. Eom.  Mapping the intellectual structure of research in decision support systems through author cocitation analysis (1971-1993) , 1996, Decis. Support Syst..

[78]  Jeretta Horn Nord,et al.  MIS research: Journal status assessment and analysis , 1995, Inf. Manag..

[79]  Bill C. Hardgrave,et al.  Forums for management information systems scholars , 1995, CACM.

[80]  L. Graham,et al.  Science in Russia and the Soviet Union: A Short History by Loren R. Graham (review) , 1993, Technology and Culture.

[81]  Jonathan K. Trower.  Publications by Researchers and Institutions in Two Top IS Journals: 1990-1994 , 1995 .

[82]  Clyde W. Holsapple,et al.  Business Computing Research Journals: A Normalized Citation Analysis , 1994, J. Manag. Inf. Syst..

[83]  Sean B. Eom.  Ranking institutional contributions to decision support systems research: a citation analysis , 1994, DATB.

[84]  Clyde W. Holsapple,et al.  Business computing system research: Structuring the field , 1994 .

[85]  Giacomo Mauro DAriano.  Identity and Control: A Structural Theory of Social Action. , 1994 .

[86]  Herman Manakyan,et al.  A citation analysis of business computing research journals , 1993, Inf. Manag..

[87]  Sean B. Eom,et al.  The intellectual structure of decision support systems (1971-1989) , 1993, Decis. Support Syst..

[88]  Sean B. Eom,et al.  Leading U.S. universities and most influential contributors in decision support systems research (1971-1989) a citation analysis , 1993, Decis. Support Syst..

[89]  Varun Grover,et al.  The evolution of empirical research in IS: A study in IS maturity , 1993, Inf. Manag..

[90]  David Blair,et al.  Communicating MIS Research: A Citation Study of Journal Influence , 1993, Inf. Process. Manag..

[91]  L. Graham,et al.  Science in Russia and the Soviet Union , 1993 .

[92]  H. White.  Identity and Control: A Structural Theory of Social Action , 1993 .

[93]  Diane Lending,et al.  Update on MIS research: a profile of leading journals and U.S. universities , 1992, DATB.

[94]  Varun Grover,et al.  Research in MIS—points of work and reference: a replication and extension of the Culnan and Swanson study , 1992, DATB.

[95]  Mark L. Gillenson,et al.  Academic Issues in MIS: Journals and Books , 1991, MIS Q..

[96]  Ravinder Nath,et al.  Productivity of management information systems researchers: Does Lotka's law apply? , 1991, Inf. Process. Manag..

[97]  Edwin Mansfield,et al.  Academic research and industrial innovation , 1991 .

[98]  Jung P. Shim,et al.  An examination of articles in the eight leading management information systems journals: 1980–1988 , 1991 .

[99]  H. Klein,et al.  Information systems research: contemporary approaches and emergent traditions , 1991 .

[100]  H. B. Eom,et al.  DSS applications development research: leading institutions and most frequent contributors (1971–1988) , 1990 .

[101]  Sang M. Lee,et al.  DSS Applications developments research: Leading institutions and most frequent contributors (1971 - April 1988) , 1990, Decis. Support Syst..

[102]  L. Leydesdorff,et al.  Dimensions of Citation Analysis , 1990 .

[103]  Michael H. MacRoberts,et al.  Problems of citation analysis: A critical review , 1989, JASIS.

[104]  Mary J. Culnan,et al.  Mapping the Intellectual Structure of MIS, 1980-1985: A Co-Citation Analysis , 1987, MIS Q..

[105]  L. Georghiou,et al.  Evaluation of Research: A Selection of Current Practices , 1987 .

[106]  Ali F. Farhoomand,et al.  Scientific progress of management information systems , 1987, DATB.

[107]  E. Burton Swanson,et al.  Research in Management Information Systems, 1980-1984: Points of Work and Reference , 1986, MIS Q..

[108]  Mary J. Culnan,et al.  The intellectual development of management information systems, 1972-1982: a co-citation analysis , 1986 .

[109]  T. Brooks.  Evidence of complex citer motivations , 1986, J. Am. Soc. Inf. Sci..

[110]  Terrence A. Brooks,et al.  Private acts and public objects: An investigation of citer motivations , 1985, J. Am. Soc. Inf. Sci..

[111]  S. Cozzens.  Comparing the Sciences: Citation Context Analysis of Papers from Neuropharmacology and the Sociology of Science , 1985 .

[112]  Gordon B. Davis,et al.  Management information systems : conceptual foundations, structure, and development , 1985 .

[113]  Douglas R. Vogel,et al.  MIS research: a profile of leading journals and universities , 1984, DATB.

[114]  M. Lynne Markus,et al.  Power, politics, and MIS implementation , 1987, CACM.

[115]  Scott Hamilton,et al.  The journal communication system for MIS research , 1982, DATB.

[116]  Scott Hamilton,et al.  Knowledge Utilization Among MIS Researchers , 1982, MIS Q..

[117]  S. Shapin.  Laboratory life. The social construction of scientific facts , 1981, Medical History.

[118]  Blaise Cronin,et al.  The Need for a Theory of citing , 1981, J. Documentation.

[119]  Susan E. Cozzens,et al.  Taking the Measure of Science: A Review of Citation Theories , 1981 .

[120]  Scott Hamilton,et al.  Communication of MIS Research: an Analysis of Journal Stratification , 1980, ICIS.

[121]  Peter G. W. Keen,et al.  Mis Research: Reference disciplines and a Cumulative Tradition , 1980, ICIS.

[122]  Henry Small,et al.  Cited Documents as Concept Symbols , 1978 .

[123]  N. Mullins,et al.  The Group Structure of Cocitation Clusters: A Comparative Study , 1977 .

[124]  G. Gilbert.  Referencing as Persuasion , 1977 .

[125]  G. Gilbert,et al.  The Transformation of Research Findings into Scientific Knowledge , 1976 .

[126]  R. Merton,et al.  The Sociology of Science: Theoretical and Empirical Investigations , 1973 .

[127]  John Law,et al.  Normative and Interpretive Sociologies of Science , 1974 .

[128]  N. Mullins,et al.  Coherent social groups in scientific change. , 1972, Science.

[129]  W. Hagstrom.  Inputs, Outputs, and the Prestige of University Science Departments , 1971 .

[130]  A. Cawkell.  Science Citation Index , 1970, Nature.

[131]  A. Pritchard,et al.  Statistical bibliography or bibliometrics , 1969 .

[132]  J. R. Cole,et al.  Scientific output and recognition: a study in the operation of the reward system in science. , 1967, American sociological review.

[133]  D. Price,et al.  Collaboration in an invisible college. , 1966, The American psychologist.

[134]  D J PRICE,et al.  NETWORKS OF SCIENTIFIC PAPERS. , 1965, Science.

[135]  Norman M Kaplan,et al.  The norms of citation behavior: Prolegomena to the footnote , 1965 .

[136]  Eugene Garfield,et al.  THE USE OF CITATION DATA IN WRITING THE HISTORY OF SCIENCE , 1964 .

[137]  E Garfield,et al.  "Science Citation Index"--A New Dimension in Indexing. , 1964, Science.

[138]  E. Garfield,et al.  Can Citation Indexing Be Automated ? , 1964 .

[139]  Eugene Garfield,et al.  Citation indexes in sociological and historical research , 1963 .

[140]  C. Best.  The internal secretion of the pancreas. , 1922, Canadian Medical Association journal.

[141]  J. Westbrook.  Identifying Significant Research. , 1960, Science.

[142]  E GARFIELD,et al.  Citation indexes for science; a new dimension in documentation through association of ideas. , 2006, Science.

[143]  Alfred J. Lotka,et al.  The frequency distribution of scientific productivity , 1926 .

[144]  Melvin Weinatoek.  Citation Indexes , 2022 .