Ranking games

The outcomes of many strategic situations such as parlor games or competitive economic scenarios are rankings of the participants, with higher ranks generally at least as desirable as lower ranks. Here we define ranking games as a class of n-player normal-form games with a payoff structure reflecting the players' von Neumann-Morgenstern preferences over their individual ranks. We investigate the computational complexity of a variety of common game-theoretic solution concepts in ranking games and deliver hardness results for iterated weak dominance and mixed Nash equilibrium when there are more than two players, and for pure Nash equilibrium when the number of players is unbounded but the game is described succinctly. This dashes hope that multi-player ranking games can be solved efficiently, despite their profound structural restrictions. Based on these findings, we provide matching upper and lower bounds for three comparative ratios, each of which relates two different solution concepts: the price of cautiousness, the mediation value, and the enforcement value.

[1]  Teresa M. Amabile,et al.  Creativity In Context: Update To The Social Psychology Of Creativity , 1996 .

[2]  Leslie M. Marx,et al.  Order Independence for Iterated Weak Dominance , 2000, Games Econ. Behav..

[3]  N. Adler,et al.  When Knowledge Wins: Transcending the Sense and Nonsense of Academic Rankings , 2009 .

[4]  P. Seglen,et al.  Education and debate , 1999, The Ethics of Public Health.

[5]  Teresa M. Amabile,et al.  Assessing the Work Environment for Creativity , 1996 .

[6]  Eric van Damme,et al.  Non-Cooperative Games , 2000 .

[7]  P. Rothwell,et al.  Reproducibility of peer review in clinical neuroscience. Is agreement between reviewers any greater than would be expected by chance alone? , 2000, Brain : a journal of neurology.

[8]  Bruno S. Frey,et al.  Economists in the PITS? , 2009, SSRN Electronic Journal.

[9]  Keki B. Irani,et al.  An Algorithmic Solution of N-Person Games , 1986, AAAI.

[10]  L. G. H. Cijan.  A polynomial algorithm in linear programming , 1979 .

[11]  M. Mahoney.  Publication prejudices: An experimental study of confirmatory bias in the peer review system , 1977, Cognitive Therapy and Research.

[12]  R. Merton.  The Matthew Effect in Science , 1968, Science.

[13]  I. Janis.  Victims of Groupthink: A psychological study of foreign-policy decisions and fiascoes. By Irving L. Janis. (Boston: Houghton Mifflin, 1972. viii + 276 pp. Map, illustrations, chart, notes, sources, bibliography, and index. Cloth, $7.95; paper $4.50.) , 1973 .

[14]  M. M. G. Fase,et al.  Notes and Communications , 2007 .

[15]  H. Browman,et al.  Factors and indices are one thing, deciding who is scholarly, why they are scholarly, and the relative value of their scholarship is something else entirely , 2008 .

[16]  Steven Vajda,et al.  The theory of games and linear programming , 1956 .

[17]  Teresa M. Amabile,et al.  Reality, intrinsic motivation, and creativity. , 1998 .

[18]  Christos H. Papadimitriou,et al.  Worst-case Equilibria , 1999, STACS.

[19]  Edwin A. Locke,et al.  Has Goal Setting Gone Wild, or Have Its Attackers Abandoned Good Scholarship? , 2009 .

[20]  Bruno S. Frey,et al.  The As-Is Journal Review Process: Let Authors Own Their Ideas. , 2007 .

[21]  P. Woit.  Not Even Wrong , 2006 .

[22]  J. Armstrong,et al.  Peer review for journals: Evidence on quality control, fairness, and innovation , 1997 .

[23]  K. Arrow,et al.  The New Palgrave Dictionary of Economics , 2020 .

[24]  David Cyranoski,et al.  Cash for papers: putting a premium on publication , 2006, Nature.

[25]  J. Harsanyi.  Oddness of the number of equilibrium points: A new proof , 1973 .

[26]  E. Swanson,et al.  Publishing in the Majors: A Comparison of Accounting, Finance, Management, and Marketing , 2002 .

[27]  B. Frey,et al.  Publishing as Prostitution? Choosing between One's Own Ideas and Academic Failure , 2002 .

[28]  M. Polanyi.  The Republic of Science: Its Political and Economic Theory , 1962 .

[29]  Jacques-François Thisse,et al.  Unilaterally competitive games , 1992 .

[30]  Georg Gottlob,et al.  Pure Nash equilibria: hard and easy games , 2003, TARK '03.

[31]  P. David,et al.  Toward a new economics of science , 1994 .

[32]  W. Espeland,et al.  Rankings and Reactivity: How Public Measures Recreate Social Worlds1 , 2007, American Journal of Sociology.

[33]  R. Aumann.  Subjectivity and Correlation in Randomized Strategies , 1974 .

[34]  Janne S. Kotiaho,et al.  Unfamiliar citations breed mistakes , 1999, Nature.

[35]  M. Dufwenberg.  Game theory. , 2011, Wiley interdisciplinary reviews. Cognitive science.

[36]  Susanne Neckermann,et al.  Awards: A View from Psychological Economics , 2008 .

[37]  Teresa M. Amabile,et al.  Reward, Intrinsic Motivation, and Creativity , 1998 .

[38]  L. Butler,et al.  Assessing university research: A plea for a balanced approach , 2007 .

[39]  E. Deci,et al.  A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. , 1999, Psychological bulletin.

[40]  J. Lane.  Let's make science metrics more scientific , 2010, Nature.

[41]  B. Frey,et al.  Motivation crowding theory , 2001 .

[42]  P. Lawrence.  The politics of publication , 2003, Nature.

[43]  S. T. Buckland,et al.  An Introduction to the Bootstrap. , 1994 .

[44]  Richard Smith,et al.  Journal accused of manipulating impact factor , 1997 .

[45]  Donald Gillies,et al.  Hempelian and Kuhnian approaches in the philosophy of medicine: the Semmelweis case. , 2005, Studies in history and philosophy of biological and biomedical sciences.

[46]  Kathleen M. Eisenhardt,et al.  Control: Organizational and Economic Approaches , 1985 .

[47]  R. Merton,et al.  The Sociology of Science: Theoretical and Empirical Investigations , 1975, Journal for the Scientific Study of Religion.

[48]  Donald S. Siegel,et al.  A Theoretical and Empirical Analysis of Journal Rankings: The Case of Formal Lists , 2000 .

[49]  Paul W. Goldberg,et al.  The complexity of computing a Nash equilibrium , 2006, STOC '06.

[50]  Bruno S. Frey,et al.  Tertium Datur: Pricing, Regulating and Intrinsic Motivation , 1992 .

[51]  R. Nelson.  The Simple Economics of Basic Scientific Research , 1959, Journal of Political Economy.

[52]  D. Cicchetti.  The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation , 1991, Behavioral and Brain Sciences.

[53]  Philip Campbell,et al.  Escape from the impact factor , 2008 .

[54]  J. E. Hirsch,et al.  An index to quantify an individual's scientific research output , 2005, Proc. Natl. Acad. Sci. USA.

[55]  Olivier de Wolf.  Optimal Strategies in n-Person Unilaterally Competitive Games , 1999 .

[56]  Henk Norde,et al.  Bimatrix games have quasi-strict equilibria , 1999, Math. Program..

[57]  Siegwart Lindenberg.  Intrinsic motivation in a new light , 2001 .

[58]  R. Posner.  From the new institutional economics to organization economics: with applications to corporate governance, government agencies, and legal institutions , 2010, Journal of Institutional Economics.

[59]  Vincent Larivière,et al.  History of the journal impact factor: Contingencies and consequences , 2009, Scientometrics.

[60]  Frederic S. Lee.  The Research Assessment Exercise, the state and the dominance of mainstream economics in British universities , 2006 .

[61]  K. Arrow.  Economic Welfare and the Allocation of Resources for Invention , 1962 .

[62]  A. Falk,et al.  The hidden costs of control , 2006 .

[63]  C. Chow,et al.  Are Articles in “Top” Management Journals Necessarily of Higher Quality? , 2007 .

[64]  Bruno S. Frey,et al.  Are more and better indicators the solution , 2009 .

[65]  Vincent Conitzer,et al.  Complexity of (iterated) dominance , 2005, EC '05.

[66]  B. Clark.  Creating entrepreneurial universities : organizational pathways of transformation , 1998 .

[67]  Wendy Nelson Espeland,et al.  The Discipline of Rankings: Tight Coupling and Organizational Change , 2009 .

[68]  J. Evans,et al.  Quotational and reference accuracy in surgical journals. A continuing peer review problem. , 1990, JAMA.

[69]  William H. Starbuck,et al.  The Production Of Knowledge , 2006 .

[70]  Kevin G. Corley,et al.  Being Good Versus Looking Good: Business School Rankings and the Circean Transformation From Substance to Image , 2002 .

[71]  Juan Miguel Campanario,et al.  UsingCitation Classics to study the incidence of serendipity in scientific discovery , 1996, Scientometrics.

[72]  Daniel M. Kane,et al.  On the complexity of two-player win-lose games , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[73]  Arthur G. Bedeian,et al.  The Manuscript Review Process , 2003 .

[74]  R. Lucas.  Econometric policy evaluation: A critique , 1976 .

[75]  Maxwell L. King,et al.  Measuring research quality using the journal impact factor, citations and ‘Ranked Journals’: blunt instruments or inspired metrics? , 2009 .

[76]  Alexander Schrijver,et al.  Theory of linear and integer programming , 1986, Wiley-Interscience series in discrete mathematics and optimization.

[77]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[78]  Margit Osterloh,et al.  Governance by Numbers. Does It Really Work in Research? , 2010 .

[79]  Juan Miguel Campanario,et al.  Peer Review for Journals as it Stands Today—Part 2 , 1998 .

[80]  Hugh Willmott,et al.  Just How Managed is the McUniversity? Craig Prichard, Hugh Willmott , 1997 .

[81]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[82]  Philip Mirowski,et al.  Science Bought and Sold , 2002 .

[83]  Stefan Thurner,et al.  Peer-review in a world with rational scientists: Toward selection of the average , 2010, 1008.4324.

[84]  Eugene Garfield,et al.  Editors are justified in asking authors to cite equivalent references from same journal , 1997 .

[85]  Roger B. Myerson,et al.  Game theory - Analysis of Conflict , 1991 .

[86]  E. Damme.  Refinements of the Nash Equilibrium Concept , 1983 .

[87]  E. Garfield.  Citation analysis as a tool in journal evaluation. , 1972, Science.

[88]  Daniel A. Levinthal,et al.  Hoping for A to Z While Rewarding Only A: Complex Organizations and Multiple Goals , 2009, Organ. Sci..

[89]  Xiaotie Deng,et al.  Settling the Complexity of Two-Player Nash Equilibrium , 2006, 2006 47th Annual IEEE Symposium on Foundations of Computer Science (FOCS'06).

[90]  Paula E. Stephan.  Science and the University: Challenges for Future Research , 2008 .

[91]  Anthony F. J. van Raan,et al.  Fatal attraction: Conceptual and methodological problems in the ranking of universities by bibliometric methods , 2005, Scientometrics.

[92]  Dirk Helbing,et al.  How to create an innovation accelerator , 2010, ArXiv.

[93]  J. Morgan,et al.  The Spite Motive and Equilibrium Behavior in Auctions , 2003 .

[94]  M. Polanyi.  The Republic of science , 1962 .

[95]  R. Aumann.  Almost Strictly Competitive Games , 1961 .

[96]  T. Kuhn,et al.  The Structure of Scientific Revolutions. , 1964 .

[97]  A. Oswald.  An Examination of the Reliability of Prestigious Scholarly Journals: Evidence and Implications for Decision-Makers , 2006, SSRN Electronic Journal.

[98]  E. Deci,et al.  Self‐determination theory and work motivation , 2005 .

[99]  Paula E. Stephan.  The Economics of Science , 1996 .

[100]  L. Butler,et al.  Explaining Australia’s increased share of ISI publications—the effects of a funding formula based on publication counts , 2003 .

[101]  Jonathan Schaeffer,et al.  Computers, Chess, and Cognition , 2011, Springer New York.

[102]  D. Campbell.  Factors relevant to the validity of experiments in social settings. , 1957, Psychological bulletin.

[103]  R. Aumann.  On the Non-transferable Utility Value: A Comment on the Roth-Shafer Examples [Values for Games without Side Payments: Some Difficulties with Current Concepts] [On the Existence and Interpretation of Value Allocations] , 1985 .

[104]  Robin P. Cubitt,et al.  Rationally Justifiable Play and the Theory of Non-cooperative Games , 1994 .

[105]  Michael Roach,et al.  A Taste for Science? PhD Scientists’ Academic Orientation and Self-Selection into Research Careers in Industry , 2010 .

[106]  B. Frey,et al.  Motivation, Knowledge Transfer, and Organizational Forms , 2000 .

[107]  Nathan R. Sturtevant,et al.  Current Challenges in Multi-player Game Search , 2004, Computers and Games.

[108]  J. Gans,et al.  How Are the Mighty Fallen: Rejected Classic Articles by Leading Economists , 1994 .

[109]  Paul R. Milgrom,et al.  Multitask Principal–Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design , 1991 .

[110]  C. Papadimitriou.  Algorithmic Game Theory: The Complexity of Finding Nash Equilibria , 2007 .

[111]  Yoav Shoham,et al.  Spiteful Bidding in Sealed-Bid Auctions , 2007, IJCAI.

[112]  F. Hayek,et al.  The Fatal Conceit: The Errors of Socialism , 1989 .

[113]  Teresa M. Amabile,et al.  How to kill creativity. , 1998, Harvard business review.

[114]  B. Frey.  Not Just for the Money: An Economic Theory of Personal Motivation , 1998 .

[115]  Peter Norvig,et al.  Artificial intelligence - a modern approach, 2nd Edition , 2003, Prentice Hall series in artificial intelligence.

[116]  Burt Perrin,et al.  Effective use and misuse of performance measurement , 1998 .

[117]  D. Eckberg,et al.  When nonreliability of reviews indicates solid science , 1991, Behavioral and Brain Sciences.

[118]  Itai Ashlagi,et al.  On the Value of Correlation , 2005, UAI.

[119]  W. Ouchi.  A Conceptual Framework for the Design of Organizational Control Mechanisms , 1979 .

[120]  Peter Taylor,et al.  Citation Statistics , 2009, ArXiv.

[121]  Michael Power,et al.  The Theory of the Audit Explosion , 2007 .

[122]  S. Albers.  Misleading Rankings of Research in Business , 2009 .

[123]  Pablo R Ros,et al.  Motivation and compensation in academic radiology. , 2004, Journal of the American College of Radiology : JACR.

[124]  J. V. Heilig,et al.  Accountability Texas-Style: The Progress and Learning of Urban Minority Students in a High-Stakes Testing Context , 2008 .

[125]  R. Monastersky.  The number that's devouring science , 2005 .

[126]  Moshe Tennenholtz,et al.  Competitive Safety Analysis: Robust Decision-Making in Multi-Agent Systems , 2002, J. Artif. Intell. Res..

[127]  Juan Miguel Campanario,et al.  Peer Review for Journals as it Stands Today—Part 1 , 1998 .

[128]  L. Khachiyan.  Polynomial algorithms in linear programming , 1980 .

[129]  Alfred Kieser,et al.  How much do specialists have to learn from each other when they jointly develop radical product innovations , 2008 .

[130]  Peter Weingart,et al.  Impact of bibliometrics upon the science system: Inadvertent consequences? , 2005, Scientometrics.

[131]  E. Damme.  On the state of the art in game theory : An interview with Robert Aumann , 1998 .

[132]  Christos H. Papadimitriou,et al.  Computing correlated equilibria in multi-player games , 2005, STOC '05.

[133]  T. Raghavan,et al.  Non-zero-sum two-person games , 2002 .

[134]  M. Schweitzer,et al.  Goals Gone Wild: The Systematic Side Effects of Overprescribing Goal Setting , 2009 .

[135]  M. V. Simkin,et al.  Copied citations create renowned papers , 2003, cond-mat/0305150.

[136]  Onn Shehory,et al.  Coalition structure generation with worst case guarantees , 2022 .

[137]  M. Osterloh,et al.  Pay for Performance in the Public Sector—Benefits and (Hidden) Costs , 2010 .

[138]  Amanda H. Goodall.  Highly Cited Leaders and the Performance of Research Universities. , 2008 .

[139]  Tim Roughgarden,et al.  Selfish routing and the price of anarchy , 2005 .

[140]  R. Eisenberger,et al.  Detrimental effects of reward. Reality or myth? , 1996, The American psychologist.

[141]  Bruno S. Frey,et al.  Motivate people with prizes , 2010 .

[142]  Lutz Bornmann,et al.  The luck of the referee draw: the effect of exchanging reviews , 2009, Learn. Publ..

[143]  Nina Mazar,et al.  Large stakes and big mistakes , 2009 .

[144]  L. Adrian,et al.  Creativity in Science , 2015 .

[145]  E. Rowland.  Theory of Games and Economic Behavior , 1946, Nature.

[146]  F. Nijhuis,et al.  Prediction of scientific performance in clinical medicine , 1990 .

[147]  J. Neumann.  Zur Theorie der Gesellschaftsspiele , 1928 .

[148]  M. Abernethy,et al.  Management control systems in research and development organizations: The role of accounting, behavior and personnel controls , 1997 .

[149]  Randall G. Holcombe,et al.  The National Research Council Ranking of Research Universities: Its Impact on Research in Economics , 2004 .

[150]  William H. Starbuck,et al.  How Much Better are the Most Prestigious Journals? The Statistics of Academic Publication , 2005, Organ. Sci..

[151]  Maurice E. Schweitzer,et al.  Goal Setting as a Motivator of Unethical Behavior , 2004 .

[152]  D. Horrobin,et al.  Peer review of grant applications: a harbinger for mediocrity in clinical research? , 1996, The Lancet.

[153]  S. Vajda,et al.  GAMES AND DECISIONS; INTRODUCTION AND CRITICAL SURVEY. , 1958 .

[154]  Salil P. Vadhan,et al.  Computational Complexity , 2005, Encyclopedia of Cryptography and Security.

[155]  B. Frey,et al.  Do Rankings Reflect Research Quality? , 2008, SSRN Electronic Journal.

[156]  V. Bush.  Science, the Endless Frontier , 1999, Science, the Endless Frontier.

[157]  G. Glass,et al.  High-stakes testing and student achievement: Does accountability pressure increase student learning? , 2006 .

[158]  Franck Laloë,et al.  Bibliometric evaluation of individual researchers: not even right... not even wrong! , 2009 .

[159]  B. Frey,et al.  Corporate Governance: What Can We Learn from Public Governance? , 2006 .

[160]  Henk F. Moed,et al.  The future of research evaluation rests with an intelligent combination of advanced metrics and transparent peer review , 2007 .

[161]  Lutz Bornmann,et al.  OPEN PEN ACCESS CCESS , 2008 .

[162]  Dan L. Worrell.  Assessing Business Scholarship: The Difficulties in Moving Beyond the Rigor-Relevance Paradigm Trap , 2009 .

[163]  R. Nelson.  The market economy, and the scientific commons , 2004 .

[164]  J. Vial,et al.  Strategically zero-sum games: The class of games whose completely mixed equilibria cannot be improved upon , 1978 .

[165]  C. Donovan.  The qualitative future of research evaluation , 2007 .

[166]  Margit Osterloh,et al.  Governing Knowledge Work:: Transactional and Transformational Solutions , 2010 .

[167]  Robert D. Tollison,et al.  Dry Holes in Economic Research , 2003 .

[168]  Heinz Maier-Leibnitz.  The measurement of quality and reputation in the world of learning , 1989 .

[169]  Arthur G. Bedeian,et al.  Peer Review and the Social Construction of Knowledge in the Management Discipline , 2004 .

[170]  P. Perakakis,et al.  The siege of science , 2008 .

[171]  A. Rustichini,et al.  Pay Enough or Don't Pay at All , 2000 .

[172]  Ernst Fehr,et al.  Fairness and Incentives in a Multi-Task Principal-Agent Model , 2004 .

[173]  Mei-Shu Lai,et al.  The unintended consequence of diabetes mellitus pay-for-performance (P4P) program in Taiwan: are patients with more comorbidities or more severe conditions likely to be excluded from the P4P program? , 2011, Health services research.

[174]  O. Sorenson,et al.  Science and the Diffusion of Knowledge , 2001 .