Does Paying Referees Expedite Reviews?: Results of a Natural Experiment

A natural experiment in an economics field journal afforded time-series observations on payments to referees for on-time reviews. The natural experiment yielded 15 months’ worth of data with no payments and about two subsequent years of data with payments. Using refereeand manuscript-specific measures as covariates, hazard models were used to gauge the effects of payments on individual referee’s review times. All models indicate statistically significant reductions in review times owing to referee payments. Reductions in review times translate into significant reductions in first-response time (FRT). Median FRT was reduced from 90 to 70 days, a 22% reduction in the presence of payments. With payments, only 1% of the FRTs exceeded six months; without payments, 16% of the FRTs exceeded six months.

[1]  Ofer H. Azar.  The Slowdown in First-Response Times of Economics Journals: Can it Be Beneficial? , 2006 .

[2]  David N. Laband.  Contribution, attribution and the allocation of intellectual property rights: economics versus agricultural economics , 2002 .

[3]  D. Hamermesh.  The Appointment-Book Problem and Commitment, with Applications to Refereeing and Medicine , 1991 .

[4]  James E. Prieger,et al.  A flexible parametric selection model for non‐normal data with application to health care usage , 2002 .

[5]  R. Dusansky,et al.  Rankings of U.S. Economics Departments , 1998 .

[6]  L. Tauer,et al.  Ranking Doctoral Programs by Journal Contributions of Recent Graduates , 1984 .

[7]  Gary W. Yohe,et al.  Current Publication Lags in Economics Journals , 1980 .

[8]  Ching-chong Lai,et al.  Is It Worthwhile to Pay Referees , 2001 .

[9]  G. M. Perry.  Ranking M.S. and Ph.D. Graduate Programs in Agricultural Economics , 1994 .

[10]  Joshua S. Gans,et al.  Why Referees Are Not Paid (Enough) , 1998 .

[11]  On the lags between submission and acceptance: are all referees created equal? , 2001 .

[12]  Ofer H. Azar.  Evolution of Social Norms with Heterogeneous Preferences: A General Model and an Application to the Academic Review Process , 2008 .

[13]  C. Freeman.  Do Economic Journals Obey Economic Prescriptions? , 2000 .

[14]  Daniel S. Hamermesh,et al.  Facts and Myths about Refereeing , 1994 .

[15]  L. Polopolus,et al.  Ranking of Agricultural Economics Departments: Influence of Regional Journals, Joint Authorship, and Self-Citations , 1988 .

[16]  D. Hamermesh,et al.  Scholarships, Citations and Salaries: Economic Rewards in Economics , 1982 .

[17]  Ofer H. Azar.  Rejections and the Importance of First Response Times , 2003 .

[18]  A. Lewbel,et al.  Semiparametric qualitative response model estimation with unknown heteroscedasticity or instrumental variables , 2000 .

[19]  Ofer H. Azar,et al.  The Review Process in Economics: Is it Too Fast? , 2005 .

[20]  L. Polopolus,et al.  Ranking of Agricultural Economics Departments by Citations , 1986 .

[21]  Glenn Ellison.  The Slowdown of the Economics Publishing Process , 2000 .

[22]  Ofer H. Azar.  The Academic Review Process: How Can We Make it More Efficient? , 2005 .

[23]  Paul R. Milgrom,et al.  Multitask Principal–Agent Analyses: Incentive Contracts, Asset Ownership, and Job Design , 1991 .

[24]  R. Merton,et al.  Patterns of evaluation in science: Institutionalisation, structure and functions of the referee system , 1971 .

[25]  A. Rustichini,et al.  Pay Enough or Don't Pay at All , 2000 .

[26]  Pravin K. Trivedi.  An Analysis of Publication Lags in Econometrics , 1993 .

[27]  B. Frey,et al.  Does pay motivate volunteers , 1999 .

[28]  A. Falk,et al.  The hidden costs of control , 2006 .

[29]  P. Trivedi,et al.  Joint and separate score tests for state dependence and unobserved heterogeneity , 1994 .

[30]  J. Hausman,et al.  Flexible parametric estimation of duration and competing risk models , 1990 .

[31]  Mark B. Stewart,et al.  A comparison of semiparametric estimators for the ordered response model , 2005, Comput. Stat. Data Anal..