论文信息 - Fallout and miss in journal peer review

Fallout and miss in journal peer review

Purpose – The authors exploit the analogy between journal peer review and information retrieval in order to quantify some imperfections of journal peer review.Design/methodology/approach – The authors define fallout rate and missing rate in order to describe quantitatively the weak papers that were accepted and the strong papers that were missed, respectively. To assess the quality of manuscripts the authors use bibliometric measures.Findings – Fallout rate and missing rate are put in relation with the hitting rate and success rate. Conclusions are drawn on what fraction of weak papers will be accepted in order to have a certain fraction of strong accepted papers.Originality/value – The paper illustrates that these curves are new in peer review research when interpreted in the information retrieval terminology.

Leo Egghe | Lutz Bornmann

[1] Leo Egghe,et al. The measures precision, recall, fallout and miss as a function of the number of retrieved documents and their mutual interrelations , 2008, Inf. Process. Manag..

[2] David M. Schultz,et al. Are three heads better than two? How the number of reviewers and editor behavior affect the rejection rate , 2010, Scientometrics.

[3] Leo Egghe. Existence theorem of the quadruple (P, R, F, M): Precision, recall, fallout and miss , 2007, Inf. Process. Manag..

[4] Leo Egghe. A universal method of information retrieval evaluation: the "missing" link M and the universal IR surface , 2004, Inf. Process. Manag..

[5] Michael McGill,et al. Introduction to Modern Information Retrieval , 1983 .

[6] Leo Egghe,et al. Journal peer review as an information retrieval process , 2012, J. Documentation.

[7] Lutz Bornmann,et al. The luck of the referee draw: the effect of exchanging reviews , 2009, Learn. Publ..

[8] Leo Egghe. Study of some Editor-in-Chief decision schemes , 2010 .

[9] Lutz Bornmann,et al. The manuscript reviewing process: Empirical research on review requests, review sequences, and decision rules in peer review , 2010 .