Understanding the Role and Methods of Meta-Analysis in IS Research

Four methods for reviewing a body of research literature – narrative review, descriptive review, vote-counting, and meta-analysis – are compared. Meta-analysis as a formalized, systematic review method is discussed in detail in terms of its history, current status, advantages, common analytic methods, and recent developments. Meta-analysis is found to be underutilized in IS. Suggestions on encouraging the use of metaanalysis in IS research and procedures recommended for meta-analysis are also provided.

[1]  Richard F. Beltramini,et al.  Meta-Analysis: Quantitative Methods for Research Synthesis , 1987 .

[2]  H OlsonMargrethe,et al.  User Involvement and MIS Success , 1984 .

[3]  Patrick Y. K. Chau.  On the use of construct reliability in MIS research: a meta-analysis , 1999, Inf. Manag..

[4]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[5]  Kenneth K. Boyer,et al.  Factors influencing the utilization of Internet purchasing in small organizations , 2003 .

[6]  Peter G. W. Keen,et al.  Mis Research: Reference disciplines and a Cumulative Tradition , 1980, ICIS.

[7]  Richard Baskerville,et al.  Information Systems as a Reference Discipline , 2002, MIS Q..

[8]  Jacob Cohen,et al.  A power primer. , 1992, Psychological bulletin.

[9]  Joel B. Greenhouse,et al.  Selection Models and the File Drawer Problem , 1988 .

[10]  Fanny Caranikas-Walker,et al.  A meta-analytical structural equations analysis of a model of employee turnover. , 1992 .

[11]  Dietmar Pfahl,et al.  Evaluating the learning effectiveness of using simulations in software project management education: results from a twice replicated experiment , 2004, Inf. Softw. Technol..

[12]  Rajiv Kohli,et al.  Measuring Information Technology Payoff: A Meta - Analysis of Structural Variables in Firm - Level Empirical Research , 2003, Inf. Syst. Res..

[13]  Mo Adam Mahmood,et al.  Variables affecting information technology end-user satisfaction: a meta-analysis of the empirical literature , 2000, Int. J. Hum. Comput. Stud..

[14]  Venkataraman Ramesh,et al.  Research in Information Systems: An Empirical Study of Diversity in the Discipline and Its Journals , 2002, J. Manag. Inf. Syst..

[15]  Brad J. Bushman,et al.  A procedure for combining sample correlation coefficients and vote counts to obtain an estimate and a confidence interval for the population correlation coefficient. , 1995 .

[16]  G. Glass.  Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[17]  John Mingers,et al.  Combining IS Research Methods: Towards a Pluralist Methodology , 2001, Inf. Syst. Res..

[18]  Allen S. Lee.  Integrating Positivist and Interpretive Approaches to Organizational Research , 1991 .

[19]  Jacob Cohen.  Statistical Power Analysis for the Behavioral Sciences , 1969, The SAGE Encyclopedia of Research Design.

[20]  Hassan M. Selim,et al.  An empirical investigation of student acceptance of course websites , 2003, Comput. Educ..

[21]  Maurice Landry,et al.  Can the field of MIS be disciplined? , 1989, CACM.

[22]  John Ingham,et al.  Why do people use information technology? A critical review of the technology acceptance model , 2003, Inf. Manag..

[23]  Allen I. Huffcutt,et al.  Psychometric properties of multisource performance ratings: A meta-analysis of subordinate, supervisor, peer, and self-ratings. , 1997 .

[24]  M. Markus.  Electronic Mail as the Medium of Managerial Choice , 1994 .

[25]  D. Eden.  Replication, Meta-Analysis, Scientific Progress, and AMJ's Publication Policy , 2002 .

[26]  D. Lewis,et al.  Quantitative methods in psychology , 1950 .

[27]  Gwenn W. Gröndal,et al.  Meta-analytic procedures for social research , 1993 .

[28]  John Schaubroeck,et al.  A meta-analysis of self-supervisor, self-peer, and peer-supervisor ratings. , 1988 .

[29]  S.-S. Liaw,et al.  Understanding user perceptions of World-wide web environments , 2002, J. Comput. Assist. Learn..

[30]  R. Tweedie,et al.  A Nonparametric “Trim and Fill” Method of Accounting for Publication Bias in Meta-Analysis , 2000 .

[31]  B. Green,et al.  Quantitative methods for literature reviews. , 1984, Annual review of psychology.

[32]  J Cohen,et al.  AIDS vaccine research. A new goal: preventing disease, not infection. , 1993, Science.

[33]  Shawn M. Fitzgerald,et al.  Using narrative literature reviews to build a scientific knowledge base. , 2001, Work.

[34]  Larry V. Hedges,et al.  How hard is hard science, how soft is soft science? The empirical cumulativeness of research. , 1987 .

[35]  Joseph Alexander Kim.  Using narrative literature in biblical theology , 1991 .

[36]  Mo Adam Mahmood,et al.  Factors Affecting Information Technology Usage: A Meta-Analysis of the Empirical Literature , 2001, J. Organ. Comput. Electron. Commer..

[37]  George Comstock,et al.  Television and the American child , 1991 .

[38]  R. Feinberg,et al.  E-tailers versus retailers , 2004 .

[39]  Prashant Palvia,et al.  Research Methodologies in MIS: An Update , 2004, Commun. Assoc. Inf. Syst..

[40]  Starr Roxanne Hiltz,et al.  An Assessment of Group Support Systems Experimental Research: Methodology and Results , 1998, J. Manag. Inf. Syst..

[41]  Barbara A. Kitchenham,et al.  Combining empirical results in software engineering , 1998, Inf. Softw. Technol..

[42]  Bonnie Kaplan,et al.  Combining Qualitative and Quantitative Methods in Information Systems Research: A Case Study , 1988, MIS Q..

[43]  John E. Hunter,et al.  Meta-Analysis: Cumulating Research Findings Across Studies Sage Publications: Beverly Hills, 1982, 176 pp. , 1986 .

[44]  Roger P. Greenberg,et al.  How Science Takes Stock: The Story of Meta-Analysis. , 1998 .

[45]  Rajeev Sharma,et al.  The Contingent Effects of Management Support and Task Interdependence on Successful Information Systems Implementation , 2003, MIS Q..

[46]  L. Hedges,et al.  Fixed- and random-effects models in meta-analysis. , 1998 .

[47]  A. Adam.  Whatever happened to information systems ethics? Caught between the devil and the deep blue sea , 2004 .

[48]  Harris Cooper,et al.  Statistically Combining Independent Studies: A Meta-Analysis of Sex Differences in Conformity Research , 1979 .

[49]  Randall M. Parker.  Threats to the Validity of Research. , 1993 .

[50]  Judith A. Hall,et al.  Interpreting and Evaluating Meta-Analysis , 1995, Evaluation & the health professions.

[51]  Prashant Palvia,et al.  Management Information Systems Research: What's There in a Methodology? , 2003, Commun. Assoc. Inf. Syst..

[52]  Susan E. Jackson,et al.  Meta-analysis analysis. , 1987 .

[53]  William R. King,et al.  A meta-analysis of the technology acceptance model , 2006, Inf. Manag..

[54]  G. Glass,et al.  Meta-analysis in social research , 1981 .

[55]  David A. Harrison,et al.  Meta-analysis, level of analysis, and best estimates of population correlations: Cautions for interpreting meta-analytic results in organizational behavior , 1999 .

[56]  P. Strassmann.  Information Payoff: The Transformation of Work in the Electronic Age , 1985 .

[57]  L. Simpson.  Report on Certain Enteric Fever Inoculation Statistics , 1904, British medical journal.

[58]  Ramon J. Aldag,et al.  Issues in Research Methodology , 1988 .

[59]  Siu L. Chow,et al.  Meta-Analysis of Pragmatic and Theoretical Research: A Critique , 1987 .

[60]  L. Hedges,et al.  Statistical Methods for Meta-Analysis , 1987 .

[61]  R. Rosenthal.  The file drawer problem and tolerance for null results , 1979 .

[62]  Blair T. Johnson,et al.  Comparison of three major meta-analytic approaches. , 1995 .

[63]  C. Begg,et al.  Publication bias : a problem in interpreting medical data , 1988 .

[64]  Mark I. H. Hwang,et al.  The use of meta-analysis in MIS research: promises and problems , 1996, DATB.

[65]  R. Rosenthal,et al.  Meta-analysis: recent developments in quantitative methods for literature reviews. , 2001, Annual review of psychology.

[66]  A. Field.  Meta-analysis of correlation coefficients: a Monte Carlo comparison of fixed- and random-effects methods. , 2001, Psychological methods.

[67]  R. Rosenthal.  Combining results of independent studies. , 1978 .

[68]  Guy G. Gable,et al.  Integrating case study and survey research methods: an example in information systems , 1994 .

[69]  Khaled El Emam,et al.  An Internally Replicated Quasi-Experimental Comparison of Checklist and Perspective-Based Reading of Code Documents , 2001, IEEE Trans. Software Eng..

[70]  Blake Ives,et al.  User Involvement and MIS Success: A Review of Research , 1984 .

[71]  Roger J. Calantone,et al.  A comparison of three models to explain shop‐bot use on the web , 2002 .