Single-blind vs Double-blind Peer Review in the Setting of Author Prestige.

Methods | This study was approved by the University of Washington institutional review board and conducted at Clinical Orthopaedics and Related Research (CORR), an orthopedic journal (2015 impact factor, 3.127; acceptance rate, 20%) that allows authors to select single-blind or double-blind peer review. Potential reviewers (based on expertise) were identified from the journal’s database, informed that a study on peer review would occur in the coming year, and allowed to opt out. To avoid influencing behavior, the details and timing of the study were not described and the trial was registered after completion. The protocol appears in the Supplement. Between June 2014 and August 2015, reviewers were randomized via random number table (1:1 ratio in blocks of 8) to receive single-blind or double-blind versions of an otherwise identical fabricated manuscript, which was putatively written by 2 past presidents of the American Academy of Orthopaedic Surgeons from prominent institutions. The manuscript described a prospective study on a nonclinical topic of broad interest (team training to improve communication and safety in the operating room). Five subtle errors were included, ranging from numerical mistakes to an error in the conclusion, to determine differences in how critically the manuscript was examined. The primary outcome was recommendation of acceptance or rejection. Grades of “accept” and “reject” were taken at face value and, for grades of “major revision” and “minor revision,” reviewers’ comments were analyzed for language recommending rejection by researchers blinded to group allocation. Secondary outcomes were the number of intentionally placed errors detected and quality scores for the Methods. Post hoc, scores for the other categories usually solicited by CORR were compared. Based on 80% power and an α of .05, 98 reviewers were needed to detect a 20% difference in acceptance rates. Statistical analysis used χ2, Fisher exact, student t, and negative binomial testing, and multivariable logistic regression (SAS [SAS Institute], version 9). Significance was defined as a 2-sided P value of less than .05.