Are We Wielding this Hammer Correctly? A Reflective Review of the Application of Cluster Analysis in Information Systems Research

Research Article VenuGopal Balijepally Prairie View A&M University vebalijepally@pvamu.edu George Mangalaraj Western Illinois University G-Mangalaraj@wiu.edu Kishen Iyengar University of Colorado at Boulder kishen.iyengar@colorado.edu Cluster analysis is a powerful statistical procedure for extricating natural configurations among the data and the populations. Cluster analysis, with its seemingly limitless power to produce groupings in any dataset, has all the trappings of a super-technique. However, the method produces clusters even in the absence of any natural structure in the data, and has no statistical basis to reject the null hypothesis that there are no natural groupings in the data. Application of cluster analysis, therefore, presupposes sound researcher judgment and responsible analysis and reporting. This paper summarizes the results of a reflective review of the application of cluster analysis in Information Systems (IS) research published in major IS outlets. Based on the analysis of 55 IS applications of cluster analysis, various deficiencies noticed in its use are discussed along with suggestions for future practice. By analyzing the results over two time periods, longitudinal trends in the application of this technique are highlighted.

[1]  Detmar W. Straub,et al.  Validating Instruments in MIS Research , 1989, MIS Q..

[2]  Ming-te Lu,et al.  Functional characteristics of commercial web sites: a longitudinal study in Hong Kong , 2004, Inf. Manag..

[3]  Wanda J. Orlikowski,et al.  The Problem of Statistical Power in MIS Research , 1989, MIS Q..

[4]  Detmar W. Straub,et al.  Structural Equation Modeling and Regression: Guidelines for Research Practice , 2000, Commun. Assoc. Inf. Syst..

[5]  Russell V. Lenth,et al.  SPSS 12.0 Statistical Procedures Companion , 2005 .

[6]  Detmar W. Straub,et al.  Normative standards for IS research , 1994, DATB.

[7]  David A. Tepper,et al.  Aligning Software Processes with Strategy , 2006 .

[8]  Robert E. Hoskisson,et al.  Strategic groups: Untested assertions and research proposals , 1990 .

[9]  Jae-Nam Lee,et al.  IT Outsourcing Strategies: Universalistic, Contingency, and Configurational Explanations of Success , 2004, Inf. Syst. Res..

[10]  E. Garfield.  The history and meaning of the journal impact factor. , 2006, JAMA.

[11]  Melody Y. Kiang,et al.  An Evaluation of Self-Organizing Map Networks as a Robust Alternative to Factor Analysis in Data Mining Applications , 2001, Inf. Syst. Res..

[12]  William R. King,et al.  An Empirical Assessment of the Organization of Transnational Information Systems , 1999, J. Manag. Inf. Syst..

[13]  Naresh K. Malhotra,et al.  Common Method Variance in IS Research: A Comparison of Alternative Approaches and a Reanalysis of Past Research , 2006, Manag. Sci..

[14]  André Hardy,et al.  An examination of procedures for determining the number of clusters in a data set , 1994 .

[15]  Lydia L. Gan,et al.  An empirical study of software piracy among tertiary institutions in Singapore , 2006, Inf. Manag..

[16]  Beverly K. Kahn,et al.  A Developmental Model of the Database Administration Function , 1985, J. Manag. Inf. Syst..

[17]  E. Huerta Arribas,et al.  Evaluation models of information technology in Spanish companies: a cluster analysis , 1999, Inf. Manag..

[18]  Thomas H. Davenport,et al.  Rigor vs. relevance revisited: response to Benbasat and Zmud , 1999 .

[19]  Shwu-Ing Wu,et al.  A comparison of the behavior of different customer clusters towards Internet bookstores , 2006, Inf. Manag..

[20]  Nikolaos A. Mylonopoulos,et al.  On site: global perceptions of IS journals , 2001, CACM.

[21]  Ana Jesús López Menéndez,et al.  A multivariate framework for the analysis of the digital divide: Evidence for the European Union-15 , 2006, Inf. Manag..

[22]  Ingoo Han,et al.  Performance measure of information systems (IS) in evolving computing environments: an empirical investigation , 2003, Inf. Manag..

[23]  Mark Keil,et al.  Understanding software project risk: a cluster analysis , 2004, Inf. Manag..

[24]  Paul Benjamin Lowry,et al.  Global Journal Prestige and Supporting Disciplines: A Scientometric Study of Information Systems Journals , 2004, J. Assoc. Inf. Syst..

[25]  W. R. Dillon,et al.  On the Use of Component Scores in the Presence of Group Structure , 1989 .

[26]  SharmaRajeev,et al.  Estimating the effect of common method variance , 2009 .

[27]  Rajiv Sabherwal,et al.  An Empirical Taxonomy of the Decision-Making Processes Concerning Strategic Applications of Information Systems , 1995, J. Manag. Inf. Syst..

[28]  B DavisGordon,et al.  An investigation of media selection among directors and managers , 1998 .

[29]  Vincent Kanade,et al.  Clustering Algorithms , 2021, Wireless RF Energy Transfer in the Massive IoT Era.

[30]  Girish N. Punj,et al.  Cluster Analysis in Marketing Research: Review and Suggestions for Application , 1983 .

[31]  Joachim Griese,et al.  Investigating the buying process for the introduction of data processing in small and medium-sized firms , 1985, Inf. Manag..

[32]  Heeseok Lee,et al.  An empirical investigation of KM styles and their effect on corporate performance , 2003, Inf. Manag..

[33]  Rachid M. Zeffane,et al.  Patterns of structural control in high and low computer user organizations , 1992, Inf. Manag..

[34]  A. Adam.  Whatever happened to information systems ethics? Caught between the devil and the deep blue sea , 2004 .

[35]  Wynne W. Chin,et al.  On the use, usefulness, and ease of use of structural equation modeling in MIS research: a note of caution , 1995 .

[36]  Carol V. Brown,et al.  IT Human Resource Management Configurations and IT Turnover: Theoretical Synthesis and Empirical Analysis , 2005, Inf. Syst. Res..

[37]  L. Delbeke.  Quasi-experimentation - design and analysis issues for field settings - cook,td, campbell,dt , 1980 .

[38]  Omar El Sawy,et al.  Absorptive Capacity Configurations in Supply Chains: Gearing for Partner-Enabled Market Knowledge Creation , 2005, MIS Q..

[39]  Mark Keil,et al.  Assimilation patterns in the use of electronic procurement innovations: A cluster analysis , 2006, Inf. Manag..

[40]  William R. King,et al.  Patterns in the organization of transnational information systems , 1998, ICIS '98.

[41]  Shintaro Okazaki,et al.  What do we know about mobile Internet adopters? A cluster analysis , 2006, Inf. Manag..

[42]  DeGross.  Errata Notes: User Heterogeneity and Its Impact on Electronic Auction Market Design: An Empirical Exploration , 2004 .

[43]  Randy V. Bradley,et al.  Information Systems Success in the Context of Different Corporate Cultural Types: An Empirical Investigation , 2006, J. Manag. Inf. Syst..

[44]  Izak Benbasat,et al.  The Identity Crisis Within the IS Discipline: Defining and Communicating the Discipline's Core Properties , 2003, MIS Q..

[45]  Joyce J. Elam,et al.  Semantic Structuring in Analyst Acquisition and Representation of Facts in Requirements Analysis , 1998, Inf. Syst. Res..

[46]  Arthur G. Money,et al.  The Quantification of Decision Support Benefits Within the Context of Value Analysis , 1988, MIS Q..

[47]  John Saunders,et al.  Assessing the Value of a Quality Assurance Certificate for Software: An Exploratory Investigation , 1989, MIS Q..

[48]  Chad Anderson,et al.  Journal Quality and Citations: Common Metrics and Considerations about Their Use , 2010 .

[49]  Quey-Jen Yeh,et al.  Threats and countermeasures for information system security: A cross-industry study , 2007, Inf. Manag..

[50]  Izak Benbasat,et al.  Empirical Research in Information Systems: The Practice of Relevance , 1999, MIS Q..

[51]  Leonid Churilov,et al.  Data Mining with Combined Use of Optimization Techniques and Self-Organizing Maps for Improving Risk Grouping Rules: Application to Prostate Cancer Patients , 2005, J. Manag. Inf. Syst..

[52]  Pairin Katerattanakul,et al.  Is information systems a reference discipline? , 2006, CACM.

[53]  Jörn-Axel Meyer,et al.  The acceptance of visual information in management , 1997, Inf. Manag..

[54]  Shaila M. Miranda,et al.  Professional Versus Political Contexts: Institutional Mitigation and the Transaction Cost Heuristic in Information Systems Outsourcing , 2006, MIS Q..

[55]  Shailendra C. Palvia,et al.  Global information technology: a meta analysis of key issues , 2002, Inf. Manag..

[56]  J. Miller,et al.  Measuring the Effectiveness of Computer-Based Information Systems in the Financial Services Sector , 1987, MIS Q..

[57]  Jong-min Choe,et al.  The effect of environmental uncertainty and strategic applications of IS on a firm's performance , 2003, Inf. Manag..

[58]  Kalle Lyytinen,et al.  Nothing At The Center?: Academic Legitimacy in the Information Systems Field , 2004, J. Assoc. Inf. Syst..

[59]  Anne P. Massey,et al.  Because Time Matters: Temporal Coordination in Global Virtual Project Teams , 2003, J. Manag. Inf. Syst..

[60]  Xin Luo,et al.  A Scientometric Investigation into the Validity of IS Journal Quality Measures , 2007, J. Assoc. Inf. Syst..

[61]  K. Peffers,et al.  Identifying and Evaluating the Universe of Outlets for Information Systems Research: Ranking the Journals , 2003 .

[62]  Margherita Pagani,et al.  Determinants of adoption of High Speed Data Services in the business market: Evidence for a combined technology acceptance model with task technology fit model , 2006, Inf. Manag..

[63]  Detmar W. Straub,et al.  Validation in Information Systems Research: A State-of-the-Art Assessment , 2001, MIS Q..

[64]  P. Sopp.  Cluster analysis. , 1996, Veterinary immunology and immunopathology.

[65]  Rajiv Sabherwal,et al.  Reconciling Variance and Process Strategies for Studying Information System Development , 1995, Inf. Syst. Res..

[66]  Detmar W. Straub,et al.  Editor's comments: turning the corner: the influence of positive thinking on the information systems field , 2009 .

[67]  Rick L. Wilson,et al.  An examination of executive information system (EIS) users , 1997, Inf. Manag..

[68]  Gordon B. Davis,et al.  An Investigation of Media Selection Among Directors and Managers: From "Self" to "Other" Orientation , 1998, MIS Q..

[69]  Varun Grover,et al.  An Empirically Derived Taxonomy of Information Technology Structure and Its Relationship to Organizational Structure , 1996, J. Manag. Inf. Syst..

[70]  G. W. Milligan,et al.  An examination of procedures for determining the number of clusters in a data set , 1985 .

[71]  Timo Saarinen,et al.  System development methodology and project success: An assessment of situational approaches , 1990, Inf. Manag..

[72]  Hemant K. Jain,et al.  Success of Data Resource Management in Distributed Environments: An Empirical Investigation , 1998, MIS Q..

[73]  Thiagarajan Ravichandran,et al.  Total Quality Management in Information Systems Development: Key Constructs and Relationships , 1999, J. Manag. Inf. Syst..

[74]  Adam LeBrocq.  Style Guide for the Journal of the Association for Information Systems , 2022 .

[75]  D. Campbell,et al.  Convergent and discriminant validation by the multitrait-multimethod matrix. , 1959, Psychological bulletin.

[76]  Essentials of Scientific Method. , 1925, Nature.

[77]  G. W. Milligan,et al.  An examination of the effect of six types of error perturbation on fifteen clustering algorithms , 1980 .

[78]  E. V. Nierop.  The introduction of the 5‐year impact factor: does it benefit statistics journals? , 2010 .

[79]  Robin S. Poston,et al.  Effective Use of Knowledge Management Systems: A Process Model of Content Ratings and Credibility Indicators , 2005, MIS Q..

[80]  Rajeev Sharma,et al.  Estimating the effect of common method variance: the method-method pair technique with an illustration from TAM research , 2009 .

[81]  Alok Gupta,et al.  GIST: A Model for Design and Management of Content and Interactivity of Customer-Centric Web Sites , 2004, MIS Q..

[82]  Louis Raymond,et al.  Ideal patterns of strategic alignment and business performance , 2004, Inf. Manag..

[83]  Varun Grover,et al.  Profiles of Strategic Information Systems Planning , 1999, Inf. Syst. Res..

[84]  Detmar W. Straub,et al.  Validation Guidelines for IS Positivist Research , 2004, Commun. Assoc. Inf. Syst..

[85]  T. Cook,et al.  Quasi-experimentation: Design & analysis issues for field settings , 1979 .

[86]  Traci Carte,et al.  In Pursuit of Moderation: Nine Common Errors and Their Solutions , 2003, MIS Q..

[87]  Sridhar P. Nerur,et al.  Revolution or Evolution: A Comparison of Object-Oriented and Structured Systems Development Methods , 2001, MIS Q..

[88]  David J. Ketchen,et al.  THE APPLICATION OF CLUSTER ANALYSIS IN STRATEGIC MANAGEMENT RESEARCH: AN ANALYSIS AND CRITIQUE , 1996 .

[89]  Thomas W. Ferratt,et al.  Are Information Systems People Different? An Investigation of How They Are and Should Be Managed , 1988, MIS Q..

[90]  A. Meyer.  What Is Strategy's Distinctive Competence? , 1991 .

[91]  Howard B. Lee,et al.  Foundations of Behavioral Research , 1973 .

[92]  J. Hartigan.  Statistical theory in clustering , 1985 .

[93]  Xianggui Qu,et al.  Multivariate Data Analysis , 2007, Technometrics.

[94]  Chun Hung Cheng,et al.  Software development cost estimation: Integrating neural network with cluster analysis , 1998, Inf. Manag..

[95]  Nirup M. Menon,et al.  Information Technology Value through Different Normative Lenses , 2000, J. Manag. Inf. Syst..

[96]  Hannu Kivijärvi,et al.  Investment in information systems and the financial performance of the firm , 1995, Inf. Manag..