Assessing Cumulative Evidence within Macro Research: Why Meta-Analysis Should Be Preferred Over Vote Counting

Understanding the conclusions a body of evidence offers involves accumulating findings. Two recent articles used vote counting to assess the evidence related to important macro theories: transaction cost theory and resource-based theory. Each concluded that its focal theory is not well supported. In contrast, recent meta-analyses of the same theories concluded that both are strongly supported. We explain why macro researchers should trust the findings of meta-analyses but not those of vote counts. A direct implication is that researchers interested in advancing transaction cost and resource-based theories need to build upon the meta-analytic evidence. A broader implication is that, as the preferred method for accumulating evidence, meta-analysis can be a catalyst for the re-evaluation of established theories and the development of new theory.

[1]  Croudace Tim,et al.  What is applied psychometrics , 2010 .

[2]  J. Steenkamp,et al.  A Review and Evaluation of Meta-Analysis Practices in Management Research , 2009 .

[3]  Gaylen N. Chandler,et al.  Firm and industry effects on firm performance: A generalization and extension for new ventures , 2009 .

[4]  Michel Lander,et al.  Structure! Agency! (And Other Quarrels): A Meta-Analysis Of Institutional Theories Of Organization , 2009 .

[5]  James G. Combs,et al.  Strategic resources and performance: a meta‐analysis , 2008 .

[6]  Xiao-Hua Zhou,et al.  Statistical Methods for Meta‐Analysis , 2008 .

[7]  Herman Aguinis,et al.  Comparison of Three Meta-Analytic Procedures for Estimating Moderating Effects of Categorical Variables , 2008 .

[8]  F. Schmidt.  Meta-Analysis , 2008 .

[9]  Herman Aguinis,et al.  First Decade of Organizational Research Methods , 2009 .

[10]  C. Armstrong,et al.  A Review of Approaches to Empirical Research on the Resource-Based View of the Firm† , 2007 .

[11]  D. Hambrick.  THE FIELD OF MANAGEMENT'S DEVOTION TO THEORY: TOO MUCH OF A GOOD THING? , 2007 .

[12]  James G. Combs,et al.  The Moderating Effect of CEO Power on the Board Compositionfirm Performance Relationship* , 2007 .

[13]  K. Eddleston,et al.  Resource Configuration in Family Firms: Linking Resources, Strategic Planning and Technological Opportunities to Performance , 2007 .

[14]  Scott L. Newbert.  Empirical research on the resource‐based view of the firm: an assessment and suggestions for future research , 2007 .

[15]  David G. Sirmon,et al.  Managing Firm Resources in Dynamic Environments to Create Value: Looking Inside the Black Box , 2007 .

[16]  Kevin G. Corley,et al.  Building Better Theory by Bridging the Quantitative-Qualitative Divide , 2006 .

[17]  Benjamin A. Campbell,et al.  Encouraging Best Practice in Quantitative Management Research: An Incomplete List of Opportunities , 2006 .

[18]  Eric W. K. Tsang.  Behavioral assumptions and theory development: the case of transaction cost economics , 2006 .

[19]  Gary L. Frazier,et al.  The Moderating Influence of Firm Market Power on the Transaction Cost Economics Model: An Empirical Test in a Forward Channel Integration Context , 2006 .

[20]  J. Steenkamp,et al.  Make, buy, or ally: A transaction cost theory meta-analysis , 2006 .

[21]  Filip Lievens,et al.  Predicting adverse impact and mean criterion performance in multistage selection. , 2006, The Journal of applied psychology.

[22]  G. Hodgson,et al.  The impact of empirical tests of transaction cost economics on the debate on the nature of the firm , 2006 .

[23]  Catherine M. Dalton,et al.  Strategic Management Studies are a Special Case for Meta-Analysis , 2005 .

[24]  Yuval Deutsch.  The Impact of Board Composition on Firms’ Critical Decisions: A Meta-Analytic Review , 2005 .

[25]  Ricky Y. K. Chan,et al.  Does the Natural-Resource-Based View of the Firm Apply in an Emerging Economy? A Survey of Foreign Invested Enterprises in China , 2005 .

[26]  Brian K. Boyd,et al.  Consequences of measurement problems in strategic management research: the case of Amihud and Lev , 2005 .

[27]  M. Hitt,et al.  Construct measurement in strategic management research: illusion or reality? , 2005 .

[28]  Steven P. Brown,et al.  On the use of beta coefficients in meta-analysis. , 2005, The Journal of applied psychology.

[29]  Steven W. Floyd,et al.  Are managers from Mars and academicians from venus? Toward an understanding of the relationship between academic quality and practical relevance , 2004 .

[30]  R. David,et al.  A systematic assessment of the empirical support for transaction cost economics , 2004 .

[31]  C. Collins,et al.  STRATEGIC HUMAN RESOURCE PRACTICES, TOP MANAGEMENT TEAM SOCIAL NETWORKS, AND FIRM PERFORMANCE: THE ROLE OF HUMAN RESOURCE PRACTICES IN CREATING ORGANIZATIONAL COMPETITIVE ADVANTAGE , 2003 .

[32]  A. Hillman,et al.  Boards of Directors and Firm Performance: Integrating Agency and Resource Dependence Perspectives , 2003 .

[33]  James G. Combs,et al.  Why Do Firms Use Franchising as an Entrepreneurial Strategy?: A Meta-Analysis , 2003 .

[34]  D. Eden.  Replication, Meta-Analysis, Scientific Progress, and AMJ's Publication Policy , 2002 .

[35]  M. Hitt,et al.  Alliance Management as a Source of Competitive Advantage , 2002 .

[36]  Timothy B. Palmer,et al.  The Role of Sampling in Strategic Management Research on Performance: A Two-Study Analysis , 2002 .

[37]  Daniel Heller,et al.  Five-factor model of personality and job satisfaction: a meta-analysis. , 2002, The Journal of applied psychology.

[38]  E. Topol,et al.  Risk of cardiovascular events associated with selective COX-2 inhibitors☆ , 2002 .

[39]  Herman Aguinis.  ESTIMATION OF SAMPLING VARIANCE OF CORRELATIONS IN META‐ANALYSIS , 2001 .

[40]  E. Topol,et al.  Risk of cardiovascular events associated with selective COX-2 inhibitors. , 2001, JAMA.

[41]  Timothy B. Heath,et al.  Structural Equations Modeling , 2001 .

[42]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[43]  Jonathan L. Johnson,et al.  Number of Directors and Financial Performance: A Meta-Analysis , 1999 .

[44]  David J. Ketchen,et al.  Organizational configurations and performance: the role of statistical power in extant research , 1999 .

[45]  Jonathan L. Johnson,et al.  META-ANALYTIC REVIEWS OF BOARD COMPOSITION, LEADERSHIP STRUCTURE, AND FINANCIAL PERFORMANCE , 1998 .

[46]  Neal Schmitt,et al.  Adverse impact and predictive efficiency of various predictor combinations , 1997 .

[47]  Christopher L. Shook,et al.  Organizational Configurations And Performance: A Meta-Analysis , 1997 .

[48]  Fred S. Switzer,et al.  Meta-analyzing the relationship between grades and job performance. , 1996 .

[49]  David J. Miller,et al.  The Resource-Based View of the Firm in Two Environments: The Hollywood Film Studios From 1936 to 1965 , 1996 .

[50]  F. Schmidt.  Statistical Significance Testing and Cumulative Knowledge in Psychology: Implications for Training of Researchers , 1996 .

[51]  T. Kuhn.  The structure of scientific revolutions, 3rd ed. , 1996 .

[52]  Jonathan L. Johnson,et al.  Boards of Directors: A Review and Research Agenda: , 1996 .

[53]  D. Ones,et al.  Theory testing: Combining psychometric meta-analysis and structural equations modeling , 1995 .

[54]  Mark A. Huselid.  The Impact of Human Resource Management Practices on Turnover, Productivity, and Corporate Financial Performance , 1995 .

[55]  Abdul A. Rasheed,et al.  Configuration Research in Strategic Management: Key Issues and Suggestions , 1993 .

[56]  F. Damanpour.  Organizational Size and Innovation , 1992 .

[57]  J. Barney.  Firm Resources and Sustained Competitive Advantage , 1991 .

[58]  S. J. Goff,et al.  Joint relation of experience and ability with job performance: Test of three hypotheses. , 1988 .

[59]  Masoud Hemmasi,et al.  Assessment of statistical power in contemporary strategy research , 1987 .

[60]  John E. Hunter,et al.  THE ECONOMIC IMPACT OF JOB SELECTION METHODS ON SIZE, PRODUCTIVITY, AND PAYROLL COSTS OF THE FEDERAL WORK FORCE: AN EMPIRICALLY BASED DEMONSTRATION , 1986 .

[61]  Harris Cooper,et al.  Scientific Guidelines for Conducting Integrative Research Reviews , 1982 .

[62]  L. Hedges,et al.  Vote-counting methods in research synthesis. , 1980 .

[63]  H. Koontz.  The Management Theory Jungle Revisited , 1980 .

[64]  H. G. Osburn,et al.  An Empirical Study of the Accuracy of Corrections for Restriction in Range Due to Explicit Selection , 1979 .

[65]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[66]  O. Williamson,et al.  Markets and Hierarchies: Analysis and Antitrust Implications. , 1977 .

[67]  K. Gergen.  Social Psychology, Science and History , 1976 .

[68]  John E. Hunter,et al.  Statistical power in criterion-related validation studies. , 1976 .

[69]  Jacob Cohen,et al.  Applied multiple regression/correlation analysis for the behavioral sciences , 1979 .

[70]  R. Light,et al.  Accumulating Evidence: Procedures for Resolving Contradictions among Different Research Studies. , 1971 .

[71]  Harold Koontz,et al.  The Management Theory Jungle , 1961 .