When Knowledge Wins: Transcending the Sense and Nonsense of Academic Rankings

“Not everything that can be counted counts, and not everything that counts can be counted.”—Albert Einstein Has university scholarship gone astray? Do our academic assessment systems reward scholarship that addresses the questions that matter most to society? Using international business as an example, we highlight the problematic nature of academic ranking systems and question if such assessments are drawing scholarship away from its fundamental purpose. We call for an immediate examination of existing ranking systems, not only as a legitimate scholarly question vis-a`-vis performance—a conceptual lens with deep roots in management research—but also because the very health and vibrancy of the field are at stake. Indeed, in light of the data presented here, which suggest that current systems are dysfunctional and potentially cause more harm than good, a temporary moratorium on rankings may be appropriate until more valid and reliable ways to assess scholarly contributions can be developed. The worldwide community of scholars, along with the global network of institutions interacting with and supporting management scholarship (such as the Academy of Management, AACSB, and Thomson Reuters Scientific) are invited to innovate and design more reliable and valid ways to assess scholarly contributions that truly promote the advancement of relevant 21st century knowledge, and likewise recognize those individuals and institutions that best fulfill the university’s fundamental purpose.

[1]  W. Powell,et al.  The iron cage revisited institutional isomorphism and collective rationality in organizational fields , 1983 .

[2]  R. Merton The unanticipated consequences of purposive social action , 1936 .

[3]  Howard S. Becker,et al.  The Fate of Idealism in Medical School , 1958 .

[4]  Alan L. Mackay,et al.  Publish or perish , 1974, Nature.

[5]  S. Kerr On the folly of rewarding A, while hoping for B. , 1975, Academy of Management journal. Academy of Management.

[6]  N. Adler International Dimensions of Organizational Behavior , 1986 .

[7]  W. Scott The Adolescence of Institutional Theory. , 1987 .

[8]  L. Zucker Institutional Theories of Organization , 1987 .

[9]  Frederick Oster,et al.  RESEARCH NOTES AND COMMUNICATIONS THE DEVELOPMENT OF STRATEGIC MANAGEMENT: JOURNAL QUALITY AND ARTICLE IMPACT , 1990 .

[10]  N. Adler,et al.  The Parochial Dinosaur: Organizational Science in a Global Context , 1991 .

[11]  Andrew C. Inkpen,et al.  An Analysis of Significant Contributions to the International Business Literature , 1991 .

[12]  Nancy J. Adler,et al.  Academic and Professional Communities of Discourse: Generating Knowledge on Transnational Human Resource Management , 1992 .

[13]  S. Ghoshal,et al.  Organization Theory and the Multinational Corporation. , 1993 .

[14]  P. M. Podsakoff,et al.  Journal Influence in the Field of Management: An Analysis Using Salancik's Index in a Dependency Network , 1994 .

[15]  Andrew C. Inkpen,et al.  An Analysis of Twenty-Five Years of Research in the Journal of International Business Studies , 1994 .

[16]  J. March Continuity and Change in Theories of Organizational Action , 1996 .

[17]  Paul M. Clarke Log On ...? , 1998 .

[18]  F. DuBois,et al.  Ranking the International Business Journals , 2000 .

[19]  Donald S. Siegel,et al.  A Theoretical and Empirical Analysis of Journal Rankings: The Case of Formal Lists , 2000 .

[20]  D. Waldron International Management , 2001 .

[21]  R. Courtney The development of strategic management , 2002 .

[22]  E. Swanson,et al.  Publishing in the Majors: A Comparison of Accounting, Finance, Management, and Marketing , 2002 .

[23]  Publishing in the Majors: A Comparison of Accounting, Finance, Management, and Marketing , 2002 .

[24]  Christina T. Fong,et al.  The End of Business Schools? Less Success Than Meets the Eye , 2002 .

[25]  Kevin G. Corley,et al.  Being Good Versus Looking Good: Business School Rankings and the Circean Transformation From Substance to Image , 2002 .

[26]  M. Seligman Positive psychology: Fundamental assumptions. , 2003 .

[27]  Mike W. Peng Identifying the big question in international business research , 2004 .

[28]  Vikas Kumar,et al.  Ranking the International Business Schools: Faculty Publication as the Measure (1) , 2004 .

[29]  The Art and Practice of Leadership Coaching: 50 Top Executive Coaches Reveal Their Secrets , 2004 .

[30]  Jeffrey Pfeffer,et al.  Why Do Bad Management Theories Persist? A Comment on Ghoshal , 2005 .

[31]  Peter J. Buckley,et al.  Regaining the edge for international business research , 2005 .

[32]  Peder Olesen Larsen,et al.  Counting methods are decisive for rankings based on publication and citation studies , 2005, Scientometrics.

[33]  F. Vermeulen On Rigor and Relevance: Fostering Dialectic Progress in Management Research , 2005 .

[34]  Wendy Nelson Espeland,et al.  Strength in Numbers? The Advantages of Multiple Rankings , 2005 .

[35]  W. Bennis,et al.  How business schools lost their way. , 2005, Harvard business review.

[36]  Daniel G. Bachrach,et al.  The influence of management journals in the 1980s and 1990s , 2005 .

[37]  Bradley L. Kirkman,et al.  International Management Research in AMJ: Our Past, Present, and Future , 2005 .

[38]  Lex Donaldson,et al.  For Positive Management Theories While Retaining Science: Reply to Ghoshal , 2005 .

[39]  Anne-Wil Harzing,et al.  Australian Research Output in Economics and Business: High Volume, Low Impact? , 2005 .

[40]  William H. Starbuck,et al.  How Much Better are the Most Prestigious Journals? The Statistics of Academic Publication , 2005, Organ. Sci..

[41]  S. Ghoshal Bad Management Theories Are Destroying Good Management Practices , 2005 .

[42]  James Testa,et al.  The Thomson Scientific journal selection process. , 2006, International microbiology : the official journal of the Spanish Society for Microbiology.

[43]  Nicholas Dew Institutional Entrepreneurship , 2006 .

[44]  Hung-Gay Fung,et al.  International business research: Trends and school rankings , 2006 .

[45]  Jeroen Bosman,et al.  Scopus reviewed and compared: the coverage and functionality of the citation database Scopus, including comparisons with Web of Science and Google Scholar , 2006 .

[46]  N. Adler The Arts & Leadership: Now That We Can Do Anything, What Will We Do? , 2006 .

[47]  Joseph Macri,et al.  Rankings Methodology for International Comparisons of Institutions and Individuals: An Application to Economics in Australia and New Zealand , 2006 .

[48]  S. Rynes,et al.  What Makes Management Research Interesting, And Why Does It Matter? , 2006 .

[49]  Mike W. Peng,et al.  Most cited articles and authors in global strategy research , 2006 .

[50]  C. Chow,et al.  Are Articles in “Top” Management Journals Necessarily of Higher Quality? , 2007 .

[51]  Stuart Macdonald,et al.  Aardvark et al.: quality journals and gamesmanship in management studies , 2007, J. Inf. Sci..

[52]  Rita Gunther McGrath No Longer a Stepchild: How the Management Field can Come into Its Own , 2007 .

[53]  A. Detsky,et al.  Academic mentoring--how to give it and how to get it. , 2007, JAMA.

[54]  Ranjay Gulati,et al.  Tent Poles, Tribalism, and Boundary Spanning: The Rigor-Relevance Debate in Management Research , 2007 .

[55]  Gordon Müller-Seitz,et al.  Positive Organizational Scholarship , 2007 .

[56]  J. Dutton,et al.  Positive Organizational Scholarship , 2007 .

[57]  William H. Starbuck,et al.  Living in Mythical Spaces , 2007 .

[58]  F. Vermeulen “I Shall Not Remain Insignificant”: Adding a Second Loop to Matter More , 2007 .

[59]  S. Rynes Academy of Management Journal Editors' Forum on Citations Editor's Foreword , 2007 .

[60]  Vincent Mangematin,et al.  Global Contests in the Production of Business Knowledge , 2008 .

[61]  Anita M. McGahan,et al.  Academic Research That Matters to Managers: On Zebras, Dogs, Lemmings, Hammers, and Turnips , 2007 .

[62]  S. Rynes Carrying Sumantra Ghoshal's Torch: Creating More Positive, Relevant, And Ecologically Valid Research , 2007 .

[63]  M. Tushman,et al.  Research and Relevance: Implications of Pasteur'S Quadrant for Doctoral Programs and Faculty Development , 2007 .

[64]  S. Macdonald,et al.  Quality journals and gamesmanship in management studies , 2008 .

[65]  Michael Sauder,et al.  Interlopers and Field Change: The Entry of U.S. News into the Field of Legal Education , 2008 .

[66]  Global Business as an Agent of World Benefit: New International Business Perspectives Leading Positive Change , 2008 .

[67]  Against the Machine: Being Human in the Age of the Electronic Mob , 2008 .

[68]  Anne-Wil Harzing,et al.  Google Scholar as a new source for citation analysis , 2008 .

[69]  Frederick P. Morgeson,et al.  Same as It Ever Was: Recognizing Stability in the BusinessWeek Rankings , 2008 .

[70]  The Arts & Leadership: Now That We Can Do Anything, What Will We Do? , 2008 .

[71]  D. Griffith,et al.  Emerging themes in international business research , 2008 .

[72]  P. Lawrence Lost in publication: how measurement harms science , 2008 .

[73]  Zemsky Robert The Rain Man Cometh—Again , 2008 .

[74]  M. Segalla Publishing in the right place or publishing the right thing: journal targeting and citations' strategies for promotion and tenure committees , 2008 .

[75]  Rosalie L. Tung,et al.  From the Editors: what makes a study sufficiently international? , 2008 .

[76]  International-business scholarship: Contributing to a broader definition of success , 2008 .

[77]  Steven H. Seggie,et al.  Prolific authors and institutions in leading international business journals , 2008 .

[78]  From Higher Aims to Hired Hands: The Social Transformation of American Business Schools and the Unfulfilled Promise of Management as a Profession , 2009 .